crystalball 0.5.0 → 0.6.0

data/README.md

@@ -1,6 +1,7 @@
 # Crystalball
 
-Crystalball is a Ruby library which implements [Regression Test Selection mechanism](https://tenderlovemaking.com/2015/02/13/predicting-test-failues.html) originally published by Aaron Patterson. Its main purpose is to select a subset of your test suite which should be run to ensure your changes didn't break anything.
+Crystalball is a Ruby library which implements [Regression Test Selection mechanism](https://tenderlovemaking.com/2015/02/13/predicting-test-failues.html) originally published by Aaron Patterson.
+Its main purpose is to select a minimal subset of your test suite which should be run to ensure your changes didn't break anything.
 
 [![Build Status](https://travis-ci.org/toptal/crystalball.svg?branch=master)](https://travis-ci.org/toptal/crystalball)
 [![Maintainability](https://api.codeclimate.com/v1/badges/c8bfc25a43a1a2ecf964/maintainability)](https://codeclimate.com/github/toptal/crystalball/maintainability)
@@ -26,171 +27,25 @@ Or install it yourself as:
 
 ## Usage
 
-1. Start MapGenerator in your `spec_helper` before you loaded any file of your app. E.g.
-    ```ruby
-    Crystalball::MapGenerator.start! do |config|
-      config.register Crystalball::MapGenerator::CoverageStrategy.new
-    end
-    ```
-1. Run your test suite on clean branch with green build. This step will generate file `execution_map.yml` in your project root
-1. Make some changes to your app code
-1. Run `bundle exec crystalball` to build a prediction and run RSpec with it. Check out [RSpec runner section](#rspec-runner) for customization details.
+Please see our [official documentation](https://toptal.github.io/crystalball/).
 
-Keep in mind that as your target branch (usually master) code changes your execution maps will become outdated, 
-so you need to regenerate execution maps regularly.
+### Versioning
 
-## Map Generator
-
-There are different map generator strategies that can (and should) be used together for better predictions. Each one has its own benefits and drawbacks, so they should be configured to best fit your needs.
-
-### CoverageStrategy
-
-Uses coverage information to detect which files are covered by the given spec (i.e. the files that, if changed, may potentially break the spec);
-To customize the way the execution detection works, pass an object that responds to #detect and returns the paths to the strategy initialization:
-
-```ruby
-# ...
-config.register Crystalball::MapGenerator::CoverageStrategy.new(MyDetector)
-```
-
-By default, the execution detector is a `Crystalball::MapGenerator::CoverageStrategy::ExecutionDetector`, which filters out the paths outside the root and converts absolute paths to relative.
-
-### AllocatedObjectsStrategy
-
-Looks for the files in which the objects allocated during the spec execution are defined. It is considerably slower than `CoverageStrategy`.
-To use this strategy, use the convenient method `.build` which takes two optional keyword arguments: `only`, used to define the classes or modules to have their descendants tracked (defaults to `[]`); and `root`, which is the path where the detection will take place (defaults to `Dir.pwd`).
-Here's an example that tracks allocation of `ActiveRecord::Base` objects:
-
-```ruby
-# ...
-config.register Crystalball::MapGenerator::AllocatedObjectsStrategy.build(only: ['ActiveRecord::Base'])
-```
-
-That method is fine for most uses, but if you need to further customize the behavior of the strategy, you can directly instantiate the class.
-
-```ruby
-# ...
-config.register Crystalball::MapGenerator::AllocatedObjectsStrategy
-  .new(execution_detector: MyCustomDetector, object_tracker: MyCustomTracker)
-```
-
-The initialization takes two keyword arguments: `execution_detector` and `object_tracker`.
-`execution_detector` must be an object that responds to `#detect` receiving a list of objects and returning the paths affected by said objects. `object_tracker` is something that responds to `#used_classes_during` which yields to the caller and returns the array of classes of objects allocated during the execution of the block.
-
-### DescribedClassStrategy
-
-This strategy will take each example that has a `described_class` (i.e. examples inside `describe` blocks of classes and not strings) and add the paths where the described class and its ancestors are defined to the case map of the example;
-
-To use it, add to your `Crystalball::MapGenerator.start!` block:
-
-```ruby
-# ...
-config.register Crystalball::MapGenerator::DescribedClassStrategy.new
-```
-
-As with `AllocatedObjectsStrategy`, you can pass a custom execution detector (an object that responds to `#detect` and returns the paths) to the initialization:
-
-```ruby
-# ...
-config.register Crystalball::MapGenerator::DescribedClassStrategy.new(MyDetector)
-```
-
-### Rails specific strategies
-
-To use Rails specific strategies you must first `require 'crystalball/rails'`.
-
-#### ActionViewStrategy
-
-This strategy patches `ActionView::Template#compile!` to map the examples to affected views. Use it as follows:
-
-```ruby
-# ...
-config.register Crystalball::MapGenerator::ActionViewStrategy.new
-```
-
-#### I18nStrategy
-
-Patches I18n to have access to the path where the locales are defined, so that those paths can be added to the case map.
-To use it, add to your config:
-
-```ruby
-# ...
-config.register Crystalball::MapGenerator::I18nStrategy.new
-```
-
-### Custom strategies
-
-You can create your own strategy and use it with the map generator. Any object that responds to `#call(case_map, example)` (where `case_map` is a `Crystalball::CaseMap` and `example` a `RSpec::Core::Example`) and augmenting its list of affected files using `case_map.push(*paths_to_files)`.
-Check out the [implementation](https://github.com/toptal/crystalball/tree/master/lib/crystalball/map_generator) of the default strategies for details.
-
-Keep in mind that all the strategies configured for the map generator will run for each example of your test suite, so it may slow down the generation process considerably.
-
-## Predictor
-
-The predictor can also be customized with different strategies:
-
-### AssociatedSpecs
-
-Needs to be configured with rules for detecting which specs should be on the prediction.
-`predictor.use Crystalball::Predictor::AssociatedSpecs.new(from: %r{models/(.*).rb}, to: "./spec/models/%s_spec.rb")`
-will add `./spec/models/foo_spec.rb` to prediction when `models/foo.rb` changes.
-This strategy does not depend on a previoulsy generated case map.
-
-### ModifiedExecutionPaths
-
-Checks the case map and the diff to see which specs are affected by the new or modified files.
-
-### ModifiedSpecs
-
-As the name implies, checks for modified specs. The scope can be modified by passing a regex as argument, which defaults to `%r{spec/.*_spec\.rb\z}`.
-This strategy does not depend on a previously generated case map.
-
-### Custom strategies
-
-As with the map generator you may define custom strategies for prediction. It must be an object that responds to `#call(diff, case_map)` (where `diff` is a `Crystalball::SourceDiff` and `case_map` is a `Crystalball::CaseMap`) and returns an array of paths.
-
-Check out [default strategies implementation](https://github.com/toptal/crystalball/tree/master/lib/crystalball/predictor) for details.
-
-## Under the hood
-
-TODO: Write good description for anyone who wants to customize behavior
-
-## Spring integration
-
-It's very easy to integrate Crystalball with [Spring](https://github.com/rails/spring). Check out [spring-commands-crystalball](https://github.com/pluff/spring-commands-crystalball) for details.
-
-## Plans
-
-1. RSpec parallel integration
-1. Map size optimization
-1. Different strategies for execution map
-1. Different strategies for failure predictor
-1. Integration for git hook
-
-## RSpec Runner
-
-There is a custom RSpec runner you can use in your development with `bundle exec crystalball` command. It builds a prediction and runs it.
-
-### Runner Configuration
-
-#### Config file
-
-Create a YAML file for the runner. Default locations are `./crystalball.yml` and `./config/crystalball.yml`. You can override config path with `CRYSTALBALL_CONFIG` env variable.
-Please check an [example of a config file](https://github.com/toptal/crystalball/blob/master/spec/fixtures/crystalball.yml) for available options
-
-#### Environment variables
-
-`CRYSTALBALL_CONFIG=path/to/crystalball.yml` if you want to override default path to config file.  
-`CRYSTALBALL_SKIP_MAP_CHECK=true` if you want to skip maps expiration period check.  
-`CRYSTALBALL_SKIP_EXAMPLES_LIMIT=true` if you want to skip examples limit check.
+We use [semantic versioning](https://semver.org/) for our [releases](https://github.com/toptal/crystalball/releases).
 
 ## Development
 
 After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
 
-To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+To install this gem onto your local machine, run `bundle exec rake install`.
 
 ## Contributing
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/toptal/crystalball. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
+Bug reports and pull requests are welcome on GitHub at https://github.com/toptal/crystalball.
+This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
+
+## License
+
+Crystalball is released under the [MIT License](https://opensource.org/licenses/MIT).
+
 

data/crystalball.gemspec

@@ -7,7 +7,7 @@ require 'crystalball/version'
 Gem::Specification.new do |spec|
   spec.name          = "crystalball"
   spec.version       = Crystalball::VERSION
-  spec.authors       = ["Pavel Shutsin"]
+  spec.authors       = ["Pavel Shutsin", "Evgenii Pecherkin", "Jaimerson Araujo"]
   spec.email         = ["publicshady@gmail.com"]
 
   spec.summary       = 'A library for RSpec regression test selection'
@@ -35,14 +35,18 @@ Gem::Specification.new do |spec|
   spec.required_ruby_version = '> 2.3.0'
 
   spec.add_development_dependency 'actionview'
+  spec.add_development_dependency 'activerecord'
   spec.add_development_dependency "bundler", "~> 1.14"
+  spec.add_development_dependency 'factory_bot'
   spec.add_development_dependency 'i18n'
+  spec.add_development_dependency 'parser'
   spec.add_development_dependency 'pry'
   spec.add_development_dependency 'pry-byebug'
   spec.add_development_dependency "rake", "~> 10.0"
   spec.add_development_dependency "rspec", "~> 3.0"
-  spec.add_development_dependency 'rubocop'
+  spec.add_development_dependency 'rubocop', ">= 0.56"
   spec.add_development_dependency 'rubocop-rspec'
   spec.add_development_dependency 'simplecov'
+  spec.add_development_dependency 'sqlite3'
   spec.add_development_dependency 'yard'
 end

data/docs/index.md

@@ -0,0 +1,44 @@
+# Crystalball
+
+Crystalball is a Ruby library which implements [Regression Test Selection mechanism](https://tenderlovemaking.com/2015/02/13/predicting-test-failues.html) originally published by Aaron Patterson. 
+Its main purpose is to select a minimal subset of your test suite which should be run to ensure your changes didn't break anything.
+
+## Installation
+
+Please check our [installation instructions](https://github.com/toptal/crystalball#installation).
+
+## Basic Usage
+
+1. Start MapGenerator in your `spec_helper` before you loaded any file of your app. E.g.
+
+        if ENV['CRYSTALBALL'] == 'true' do
+          Crystalball::MapGenerator.start! do |config|
+            config.register Crystalball::MapGenerator::CoverageStrategy.new
+          end
+        end
+
+1. Run your test suite with Crystaball enabled on clean master branch with green build. `CRYSTALBALL=true bundle exec rspec .` This step will generate file `tmp/crystalball_data.yml` in your project root. This file contains useful profiling data for Crystalball.
+1. Make some changes to your app code
+1. Run `bundle exec crystalball` to build a prediction and run RSpec with it. Check out [RSpec runner section](runner.md) for customization details.
+
+Keep in mind that as your target branch (usually master) code changes your execution maps will become outdated, 
+so you need to regenerate execution maps regularly.
+
+## Advanced Usage
+
+Crystalball workflow can be divided into 2 parts. 
+1. Full build profiling where Crystalball gathers some data about your RSpec suite for later use in predictions. This is where map generators do their job.
+2. Actual predicting where Crystalball uses profiling info from step above and tries to get best prediction possible. This is where predictors do their job.
+
+Both of these steps can be heavily customized and enchanted based on your project specifics and your needs.
+
+You might want to check:
+
+* [map generators docs](map_generators.md) for details related to suite profiling.
+* [predictors docs](predictors.md) for details related to actual prediction.
+* [runner docs](runner.md) for runner configuration details. 
+
+
+## Spring integration
+
+It's very easy to integrate Crystalball with [Spring](https://github.com/rails/spring). Check out [spring-commands-crystalball](https://github.com/pluff/spring-commands-crystalball) for details.

data/docs/map_generators.md

@@ -0,0 +1,149 @@
+# Map generators
+
+## Execution Map Generator
+
+There are different map generator strategies that can (and should) be used together for better predictions. Each one has its own benefits and drawbacks, so they should be configured to best fit your needs.
+
+### Custom map file name
+
+You can customize resulting map filename with `map_storage_path` value. E.g.
+```ruby
+Crystalball::MapGenerator.start! do |config|
+  #...
+  config.map_storage_path = "execution_map_#{ENV['TEST_ENV_NUMBER'].to_i}.yml"
+end
+```
+
+### CoverageStrategy
+
+Uses coverage information to detect which files are covered by the given spec (i.e. the files that, if changed, may potentially break the spec);
+To customize the way the execution detection works, pass an object that responds to #detect and returns the paths to the strategy initialization:
+
+```ruby
+Crystalball::MapGenerator.start! do |config|
+  #...
+  config.register Crystalball::MapGenerator::CoverageStrategy.new(my_detector)
+end
+```
+
+By default, the execution detector is a `Crystalball::MapGenerator::CoverageStrategy::ExecutionDetector`, which filters out the paths outside of the project root and converts absolute paths to relative.
+
+### AllocatedObjectsStrategy
+
+Looks for the files in which the objects allocated during the spec execution are defined. It is considerably slower than `CoverageStrategy`.
+To use this strategy, use the convenient method `.build` which takes two optional keyword arguments: `only`, used to define the classes or modules to have their descendants tracked (defaults to `[]`); and `root`, which is the path where the detection will take place (defaults to `Dir.pwd`).
+Here's an example that tracks allocation of `ActiveRecord::Base` objects:
+
+```ruby
+Crystalball::MapGenerator.start! do |config|
+  #...
+  config.register Crystalball::MapGenerator::AllocatedObjectsStrategy.build(only: ['ActiveRecord::Base'])
+end
+```
+
+That method is fine for most uses, but if you need to further customize the behavior of the strategy, you can directly instantiate the class.
+
+```ruby
+Crystalball::MapGenerator.start! do |config|
+  #...
+  config.register Crystalball::MapGenerator::AllocatedObjectsStrategy
+    .new(execution_detector: my_detector, object_tracker: my_tracker)
+end
+```
+
+The initialization takes two keyword arguments: `execution_detector` and `object_tracker`.
+`execution_detector` must be an object that responds to `#detect` receiving a list of objects and returning the paths affected by said objects. `object_tracker` is something that responds to `#used_classes_during` which yields to the caller and returns the array of classes of objects allocated during the execution of the block.
+
+### DescribedClassStrategy
+
+This strategy will take each example that has a `described_class` (i.e. examples inside `describe` blocks of classes and not strings) and add the paths where the described class and its ancestors are defined to the example group map of the example;
+
+To use it, add to your `Crystalball::MapGenerator.start!` block:
+
+```ruby
+Crystalball::MapGenerator.start! do |config|
+  #...
+  config.register Crystalball::MapGenerator::DescribedClassStrategy.new
+end
+```
+
+As with `AllocatedObjectsStrategy`, you can pass a custom execution detector (an object that responds to `#detect` and returns the paths) to the initialization:
+
+```ruby
+Crystalball::MapGenerator.start! do |config|
+  #...
+  config.register Crystalball::MapGenerator::DescribedClassStrategy.new(my_detector)
+end
+```
+
+### ParserStrategy
+
+The `ParserStrategy`, as the name suggests parses the files in order to detect which files are affected by an example.
+It works by first parsing all (`.rb`) files that match the given pattern under the configured root directory (defaults to current directory) to collect the constants definition paths.
+Then, when each example is executed, the used files of the current example group map are parsed to check for method calls to those constants. For that reason, `ParserStrategy` **only works when used with other strategies and is placed at the end of the strategies list**.
+
+To use it, add the `parser` gem to your `Gemfile` and:
+
+```ruby
+require 'crystalball/map_generator/parser_strategy'
+Crystalball::MapGenerator.start! do |config|
+  #...
+  config.register Crystalball::MapGenerator::ParserStrategy.new(pattern: /\A(app)|(lib)/)
+end
+```
+
+### ActionViewStrategy
+
+To use Rails specific strategies you must first `require 'crystalball/rails'`.
+This strategy patches `ActionView::Template#compile!` to map the examples to affected views. Use it as follows:
+
+```ruby
+Crystalball::MapGenerator.start! do |config|
+  #...
+  config.register Crystalball::MapGenerator::ActionViewStrategy.new
+end 
+```
+
+### I18nStrategy
+
+To use Rails specific strategies you must first `require 'crystalball/rails'`.
+Patches I18n to have access to the path where the locales are defined, so that those paths can be added to the example group map.
+To use it, add to your config:
+
+```ruby
+Crystalball::MapGenerator.start! do |config|
+  #...
+  config.register Crystalball::MapGenerator::I18nStrategy.new
+end
+```
+
+### FactoryBotStrategy
+
+Tracks which factories were used during the example and add files with corresponding definitions to the example group map.
+To use it, add to your config:
+```ruby
+Crystalball::MapGenerator.start! do |config|
+  #...
+  config.register Crystalball::MapGenerator::FactoryBotStrategy.new
+end
+```
+
+### Custom strategies
+
+You can create your own strategy and use it with the map generator. Any object that responds to `#call(example_group_map, example)` (where `example_group_map` is a `Crystalball::ExampleGroupMap` and `example` a `RSpec::Core::Example`) and augmenting its list of used files using `example_group_map.push(*paths_to_files)`.
+Check out the [implementation](https://github.com/toptal/crystalball/tree/master/lib/crystalball/map_generator) of the default strategies for examples.
+
+Keep in mind that all the strategies configured for the map generator will run for each example of your test suite, so it may slow down the generation process considerably.
+
+## TablesMapGenerator
+
+TablesMapGenerator is a separate map generator for Rails applications. It collects information about tables-to-models mapping and stores it in a file. The file is used by `Crystalball::Rails::Predictor::ModifiedSchema`.
+Use `Crystalball::Rails::TablesMapGenerator.start!` to start it.
+
+By default TablesMapGenerator will generate `tables_map.yml` file. You can customize this behavior by setting `map_storage_path` variable:
+```ruby
+Crystalball::TablesMapGenerator.start! do |config|
+  #...
+  config.map_storage_path = 'my_custom_tables_map_name.yml'
+end
+```

data/docs/predictors.md

@@ -0,0 +1,75 @@
+# Predictors
+
+## Basic usage
+
+[By default](https://github.com/toptal/crystalball/blob/master/lib/crystalball/rspec/runner/configuration.rb) Crystalball uses `Crystalball::RSpec::StandardPredictionBuilder` which uses
+just two strategies for prediction [ModifiedExecutionPaths](#ModifiedExecutionPaths) and [ModifiedSpecs](#ModifiedSpecs).
+This is more than enough for first time use.
+
+## Advanced usage
+
+If you want to squeeze out the most out of Crystalball you might want to add additional available prediction strategies or even add your own.
+To do that you should create a class inherited from `Crystalball::RSpec::PredictionBuilder` and overload `predictor` method with your custom predictor setup similar
+to what we have in [StandardPredictionBuilder](https://github.com/toptal/crystalball/blob/master/lib/crystalball/rspec/standard_prediction_builder.rb)
+
+E.g. 
+```ruby
+class MyPredictionBuilder < Crystalball::RSpec::PredictionBuilder
+    def predictor
+      super do |p|
+        p.use Crystalball::Predictor::ModifiedSpecs.new
+        p.use Crystalball::Predictor::ModifiedExecutionPaths.new
+        p.use Crystalball::Predictor::ModifiedSupportSpecs.new
+      end
+    end
+end
+
+```
+creates a predictor with additional `ModifiedSupportSpecs` strategy enabled.
+
+You also must let our runner know about your new prediction builder by adding it to configuration. Please see [runner configuration](runner.md) page for details.
+
+### Strategies
+
+#### AssociatedSpecs
+
+Needs to be configured with rules for detecting which specs should be on the prediction.
+```ruby
+predictor.use Crystalball::Predictor::AssociatedSpecs.new(
+  from: %r{models/(.*).rb}, 
+  to: "./spec/models/%s_spec.rb"
+  )
+```
+will add `./spec/models/foo_spec.rb` to prediction when `models/foo.rb` changes.
+This strategy does not depend on a previously generated example group map.
+
+#### ModifiedExecutionPaths
+
+Checks the example group map and the diff to see which specs are affected by the new or modified files.
+
+#### ModifiedSpecs
+
+As the name implies, checks for modified specs. The scope can be modified by passing a regex as argument, which defaults to `%r{spec/.*_spec\.rb\z}`.
+This strategy does not depend on a previously generated example group map.
+
+#### ModifiedSupportSpecs
+
+Checks for modified support files used in specs and predicts full spec file. The scope can be modified by passing a regex as argument, which defaults to `%r{spec/support/.*\.rb\z}`.
+Mostly usable for shared_contexts and shared_examples.
+
+#### ModifiedSchema
+
+Checks for modified db schema in rails application. You need to specify a path to a file with tables map generated by `TablesMapGenerator`. It checks schema diff to see which models are affected by modified tables and which specs are affected by this models.
+```ruby
+predictor.use Crystalball::Rails::Predictor::ModifiedSchema.new(
+  tables_map_path: './tables_map.yml'
+  )
+```
+
+_**Note**_: You may meet a warning like "WARNING: there are no model files for changed table ...". Usually, such tables are leftovers or a relation table for `has-and-belongs-to-many` associations. For the first case - nothing to worry about. For the second case - it means you want to change the default relation table.
+
+### Custom strategies
+
+As with the map generator you may define custom strategies for prediction. It must be an object that responds to `#call(diff, example_group_map)` (where `diff` is a `Crystalball::SourceDiff` and `example_group_map` is a `Crystalball::ExampleGroupMap`) and returns an array of paths.
+
+Check out [default strategies implementation](https://github.com/toptal/crystalball/tree/master/lib/crystalball/predictor) for examples.