crystalball 0.5.0

data/README.md

@@ -0,0 +1,196 @@
+# Crystalball
+
+Crystalball is a Ruby library which implements [Regression Test Selection mechanism](https://tenderlovemaking.com/2015/02/13/predicting-test-failues.html) originally published by Aaron Patterson. Its main purpose is to select a subset of your test suite which should be run to ensure your changes didn't break anything.
+
+[![Build Status](https://travis-ci.org/toptal/crystalball.svg?branch=master)](https://travis-ci.org/toptal/crystalball)
+[![Maintainability](https://api.codeclimate.com/v1/badges/c8bfc25a43a1a2ecf964/maintainability)](https://codeclimate.com/github/toptal/crystalball/maintainability)
+[![Test Coverage](https://api.codeclimate.com/v1/badges/c8bfc25a43a1a2ecf964/test_coverage)](https://codeclimate.com/github/toptal/crystalball/test_coverage)
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+group :test do
+  gem 'crystalball'
+end
+```
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install crystalball
+
+## Usage
+
+1. Start MapGenerator in your `spec_helper` before you loaded any file of your app. E.g.
+    ```ruby
+    Crystalball::MapGenerator.start! do |config|
+      config.register Crystalball::MapGenerator::CoverageStrategy.new
+    end
+    ```
+1. Run your test suite on clean branch with green build. This step will generate file `execution_map.yml` in your project root
+1. Make some changes to your app code
+1. Run `bundle exec crystalball` to build a prediction and run RSpec with it. Check out [RSpec runner section](#rspec-runner) for customization details.
+
+Keep in mind that as your target branch (usually master) code changes your execution maps will become outdated, 
+so you need to regenerate execution maps regularly.
+
+## Map Generator
+
+There are different map generator strategies that can (and should) be used together for better predictions. Each one has its own benefits and drawbacks, so they should be configured to best fit your needs.
+
+### CoverageStrategy
+
+Uses coverage information to detect which files are covered by the given spec (i.e. the files that, if changed, may potentially break the spec);
+To customize the way the execution detection works, pass an object that responds to #detect and returns the paths to the strategy initialization:
+
+```ruby
+# ...
+config.register Crystalball::MapGenerator::CoverageStrategy.new(MyDetector)
+```
+
+By default, the execution detector is a `Crystalball::MapGenerator::CoverageStrategy::ExecutionDetector`, which filters out the paths outside the root and converts absolute paths to relative.
+
+### AllocatedObjectsStrategy
+
+Looks for the files in which the objects allocated during the spec execution are defined. It is considerably slower than `CoverageStrategy`.
+To use this strategy, use the convenient method `.build` which takes two optional keyword arguments: `only`, used to define the classes or modules to have their descendants tracked (defaults to `[]`); and `root`, which is the path where the detection will take place (defaults to `Dir.pwd`).
+Here's an example that tracks allocation of `ActiveRecord::Base` objects:
+
+```ruby
+# ...
+config.register Crystalball::MapGenerator::AllocatedObjectsStrategy.build(only: ['ActiveRecord::Base'])
+```
+
+That method is fine for most uses, but if you need to further customize the behavior of the strategy, you can directly instantiate the class.
+
+```ruby
+# ...
+config.register Crystalball::MapGenerator::AllocatedObjectsStrategy
+  .new(execution_detector: MyCustomDetector, object_tracker: MyCustomTracker)
+```
+
+The initialization takes two keyword arguments: `execution_detector` and `object_tracker`.
+`execution_detector` must be an object that responds to `#detect` receiving a list of objects and returning the paths affected by said objects. `object_tracker` is something that responds to `#used_classes_during` which yields to the caller and returns the array of classes of objects allocated during the execution of the block.
+
+### DescribedClassStrategy
+
+This strategy will take each example that has a `described_class` (i.e. examples inside `describe` blocks of classes and not strings) and add the paths where the described class and its ancestors are defined to the case map of the example;
+
+To use it, add to your `Crystalball::MapGenerator.start!` block:
+
+```ruby
+# ...
+config.register Crystalball::MapGenerator::DescribedClassStrategy.new
+```
+
+As with `AllocatedObjectsStrategy`, you can pass a custom execution detector (an object that responds to `#detect` and returns the paths) to the initialization:
+
+```ruby
+# ...
+config.register Crystalball::MapGenerator::DescribedClassStrategy.new(MyDetector)
+```
+
+### Rails specific strategies
+
+To use Rails specific strategies you must first `require 'crystalball/rails'`.
+
+#### ActionViewStrategy
+
+This strategy patches `ActionView::Template#compile!` to map the examples to affected views. Use it as follows:
+
+```ruby
+# ...
+config.register Crystalball::MapGenerator::ActionViewStrategy.new
+```
+
+#### I18nStrategy
+
+Patches I18n to have access to the path where the locales are defined, so that those paths can be added to the case map.
+To use it, add to your config:
+
+```ruby
+# ...
+config.register Crystalball::MapGenerator::I18nStrategy.new
+```
+
+### Custom strategies
+
+You can create your own strategy and use it with the map generator. Any object that responds to `#call(case_map, example)` (where `case_map` is a `Crystalball::CaseMap` and `example` a `RSpec::Core::Example`) and augmenting its list of affected files using `case_map.push(*paths_to_files)`.
+Check out the [implementation](https://github.com/toptal/crystalball/tree/master/lib/crystalball/map_generator) of the default strategies for details.
+
+Keep in mind that all the strategies configured for the map generator will run for each example of your test suite, so it may slow down the generation process considerably.
+
+## Predictor
+
+The predictor can also be customized with different strategies:
+
+### AssociatedSpecs
+
+Needs to be configured with rules for detecting which specs should be on the prediction.
+`predictor.use Crystalball::Predictor::AssociatedSpecs.new(from: %r{models/(.*).rb}, to: "./spec/models/%s_spec.rb")`
+will add `./spec/models/foo_spec.rb` to prediction when `models/foo.rb` changes.
+This strategy does not depend on a previoulsy generated case map.
+
+### ModifiedExecutionPaths
+
+Checks the case map and the diff to see which specs are affected by the new or modified files.
+
+### ModifiedSpecs
+
+As the name implies, checks for modified specs. The scope can be modified by passing a regex as argument, which defaults to `%r{spec/.*_spec\.rb\z}`.
+This strategy does not depend on a previously generated case map.
+
+### Custom strategies
+
+As with the map generator you may define custom strategies for prediction. It must be an object that responds to `#call(diff, case_map)` (where `diff` is a `Crystalball::SourceDiff` and `case_map` is a `Crystalball::CaseMap`) and returns an array of paths.
+
+Check out [default strategies implementation](https://github.com/toptal/crystalball/tree/master/lib/crystalball/predictor) for details.
+
+## Under the hood
+
+TODO: Write good description for anyone who wants to customize behavior
+
+## Spring integration
+
+It's very easy to integrate Crystalball with [Spring](https://github.com/rails/spring). Check out [spring-commands-crystalball](https://github.com/pluff/spring-commands-crystalball) for details.
+
+## Plans
+
+1. RSpec parallel integration
+1. Map size optimization
+1. Different strategies for execution map
+1. Different strategies for failure predictor
+1. Integration for git hook
+
+## RSpec Runner
+
+There is a custom RSpec runner you can use in your development with `bundle exec crystalball` command. It builds a prediction and runs it.
+
+### Runner Configuration
+
+#### Config file
+
+Create a YAML file for the runner. Default locations are `./crystalball.yml` and `./config/crystalball.yml`. You can override config path with `CRYSTALBALL_CONFIG` env variable.
+Please check an [example of a config file](https://github.com/toptal/crystalball/blob/master/spec/fixtures/crystalball.yml) for available options
+
+#### Environment variables
+
+`CRYSTALBALL_CONFIG=path/to/crystalball.yml` if you want to override default path to config file.  
+`CRYSTALBALL_SKIP_MAP_CHECK=true` if you want to skip maps expiration period check.  
+`CRYSTALBALL_SKIP_EXAMPLES_LIMIT=true` if you want to skip examples limit check.
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/toptal/crystalball. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
+

data/Rakefile

@@ -0,0 +1,8 @@
+# frozen_string_literal: true
+
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+
+RSpec::Core::RakeTask.new(:spec)
+
+task default: :spec

data/bin/console

@@ -0,0 +1,15 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require "bundler/setup"
+require "crystalball"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+# require "pry"
+# Pry.start
+
+require "irb"
+IRB.start(__FILE__)

data/bin/crystalball

@@ -0,0 +1,5 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require 'crystalball'
+Crystalball::RSpec::Runner.invoke

data/bin/setup

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Do any other automated setup that you need to do here

data/crystalball.gemspec

@@ -0,0 +1,48 @@
+# frozen_string_literal: true
+
+lib = File.expand_path('lib', __dir__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'crystalball/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = "crystalball"
+  spec.version       = Crystalball::VERSION
+  spec.authors       = ["Pavel Shutsin"]
+  spec.email         = ["publicshady@gmail.com"]
+
+  spec.summary       = 'A library for RSpec regression test selection'
+  spec.description   = 'Provides simple way to integrate regression test selection approach to your RSpec test suite'
+  spec.homepage      = 'https://github.com/toptal/crystalball'
+
+  # Prevent pushing this gem to RubyGems.org. To allow pushes either set the 'allowed_push_host'
+  # to allow pushing to a single host or delete this section to allow pushing to any host.
+  if spec.respond_to?(:metadata)
+    spec.metadata['allowed_push_host'] = "https://rubygems.org"
+  else
+    raise "RubyGems 2.0 or newer is required to protect against " \
+      "public gem pushes."
+  end
+
+  spec.files = `git ls-files -z`.split("\x0").reject do |f|
+    f.match(%r{^(test|spec|features)/})
+  end
+  spec.bindir        = "bin"
+  spec.executables   = [File.basename('bin/crystalball')]
+  spec.require_paths = ["lib"]
+
+  spec.add_dependency 'git'
+
+  spec.required_ruby_version = '> 2.3.0'
+
+  spec.add_development_dependency 'actionview'
+  spec.add_development_dependency "bundler", "~> 1.14"
+  spec.add_development_dependency 'i18n'
+  spec.add_development_dependency 'pry'
+  spec.add_development_dependency 'pry-byebug'
+  spec.add_development_dependency "rake", "~> 10.0"
+  spec.add_development_dependency "rspec", "~> 3.0"
+  spec.add_development_dependency 'rubocop'
+  spec.add_development_dependency 'rubocop-rspec'
+  spec.add_development_dependency 'simplecov'
+  spec.add_development_dependency 'yard'
+end

data/lib/crystalball.rb

@@ -0,0 +1,38 @@
+# frozen_string_literal: true
+
+require 'crystalball/git_repo'
+require 'crystalball/rspec/prediction_builder'
+require 'crystalball/rspec/runner'
+require 'crystalball/prediction'
+require 'crystalball/predictor'
+require 'crystalball/predictor/modified_execution_paths'
+require 'crystalball/predictor/modified_specs'
+require 'crystalball/predictor/associated_specs'
+require 'crystalball/case_map'
+require 'crystalball/execution_map'
+require 'crystalball/map_generator'
+require 'crystalball/map_generator/configuration'
+require 'crystalball/map_generator/coverage_strategy'
+require 'crystalball/map_generator/allocated_objects_strategy'
+require 'crystalball/map_generator/described_class_strategy'
+require 'crystalball/map_storage/yaml_storage'
+require 'crystalball/version'
+
+# Main module for the library
+module Crystalball
+  # Prints the list of specs which might fail
+  #
+  # @param [String] workdir - path to the root directory of repository (usually contains .git folder inside). Default: current directory
+  # @param [String] map_path - path to the execution map. Default: execution_map.yml
+  # @param [Proc] block - used to configure predictors
+  #
+  # @example
+  #   Crystalball.foresee do |predictor|
+  #     predictor.use Crystalball::Predictor::ModifiedExecutionPaths.new
+  #     predictor.use Crystalball::Predictor::ModifiedSpecs.new
+  #   end
+  def self.foresee(workdir: '.', map_path: 'execution_map.yml', &block)
+    map = MapStorage::YAMLStorage.load(Pathname(map_path))
+    Predictor.new(map, GitRepo.open(Pathname(workdir)), from: map.commit, &block).prediction.compact
+  end
+end

data/lib/crystalball/case_map.rb

@@ -0,0 +1,19 @@
+# frozen_string_literal: true
+
+module Crystalball
+  # Data object to store execution map for specific example
+  class CaseMap
+    attr_reader :uid, :file_path, :affected_files
+    extend Forwardable
+
+    delegate %i[push] => :affected_files
+
+    # @param [String] example - id of example
+    # @param [Array<String>] affected_files - list of files affected by example
+    def initialize(example, affected_files = [])
+      @uid = example.id
+      @file_path = example.file_path
+      @affected_files = affected_files
+    end
+  end
+end

data/lib/crystalball/execution_map.rb

@@ -0,0 +1,55 @@
+# frozen_string_literal: true
+
+module Crystalball
+  # Storage for execution map
+  class ExecutionMap
+    extend Forwardable
+
+    # Simple data object for map metadata information
+    class Metadata
+      attr_accessor :commit, :type, :version
+
+      # @param [String] commit - SHA of commit
+      # @param [String] type - type of execution map
+      # @param [Numeric] version - map generator version number
+      def initialize(commit: nil, type: nil, version: nil)
+        @commit = commit
+        @type = type
+        @version = version
+      end
+
+      def to_h
+        {type: type, commit: commit, version: version}
+      end
+    end
+
+    attr_reader :cases, :metadata
+
+    delegate %i[commit commit= version version=] => :metadata
+    delegate %i[size] => :cases
+
+    # @param [Hash] metadata - add or override metadata of execution map
+    # @param [Hash] cases - initial list of cases
+    def initialize(metadata: {}, cases: {})
+      @cases = cases
+
+      @metadata = Metadata.new(type: self.class.name, **metadata)
+    end
+
+    # Adds case map to the list
+    #
+    # @param [Crystalball::CaseMap] case_map
+    def <<(case_map)
+      cases[case_map.uid] = case_map.affected_files.uniq
+    end
+
+    # Remove all cases
+    def clear!
+      self.cases = {}
+    end
+
+    private
+
+    attr_writer :cases, :metadata
+  end
+end

data/lib/crystalball/git_repo.rb

@@ -0,0 +1,58 @@
+# frozen_string_literal: true
+
+require 'git'
+require 'crystalball/source_diff'
+
+module Crystalball
+  # Wrapper class representing Git repository
+  class GitRepo
+    attr_reader :repo_path
+
+    class << self
+      # @return [Crystalball::GitRepo] instance for given path
+      def open(repo_path)
+        path = Pathname(repo_path)
+        new(path) if exists?(path)
+      end
+
+      # Check if given path is under git control (contains .git folder)
+      def exists?(path)
+        path.join('.git').directory?
+      end
+    end
+
+    # @param [Pathname] repo_path path to repository root folder
+    def initialize(repo_path)
+      @repo_path = repo_path
+    end
+
+    # Check if repository has no uncommitted changes
+    def pristine?
+      diff.empty?
+    end
+
+    # Proxy all unknown calls to `Git` object
+    def method_missing(method, *args, &block)
+      repo.public_send(method, *args, &block) || super
+    end
+
+    def respond_to_missing?(method, *)
+      repo.respond_to?(method, false) || super
+    end
+
+    # Creates diff
+    #
+    # @param [String] from starting commit to build a diff. Default: HEAD
+    # @param [String] to ending commit to build a diff. Default: nil, will build diff of uncommitted changes
+    # @return [SourceDiff]
+    def diff(from = 'HEAD', to = nil)
+      SourceDiff.new(repo.diff(from, to))
+    end
+
+    private
+
+    def repo
+      @repo ||= Git.open(repo_path)
+    end
+  end
+end