rspec_n 1.0.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: 779f827fe4de372504a3fa7c5d4168914b4a0c3d51c05f7e441cf94ecd6953ba
4
+ data.tar.gz: 928f1f99ec404b2f3a65d07542521658dfb1296e0ed5e9e5623f256b22789db8
5
+ SHA512:
6
+ metadata.gz: b5f002fde7c4a9ad096a17abfe21a836efa53a9ce42c3d8ad94de3df68213455555f2fe5e21bd808f3a883cfa8591f9401bb925adc3e3afe5e084ccd290b2280
7
+ data.tar.gz: 48060eb007f625e03ab00e498b2c5cdf2fd365b59fb658978549a527b438bc000f988c875ff960e4f5618dd4804627b06bf1cf9852fc5d4fe4e31b10c5dc4133
data/.gitignore ADDED
@@ -0,0 +1,13 @@
1
+ /.bundle/
2
+ /.yardoc
3
+ /_yardoc/
4
+ /coverage/
5
+ /doc/
6
+ /pkg/
7
+ /spec/reports/
8
+ /tmp/
9
+ /rubocop/
10
+
11
+ # rspec failure tracking
12
+ .rspec_status
13
+ Gemfile.lock
data/.rspec ADDED
@@ -0,0 +1,3 @@
1
+ --format progress
2
+ --color
3
+ --require spec_helper
data/.rubocop.yml ADDED
@@ -0,0 +1,5 @@
1
+ inherit_gem:
2
+ rubocop_plus: config/rubocop.yml
3
+
4
+ Style/Documentation:
5
+ Enabled: false
data/.travis.yml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ sudo: false
3
+ language: ruby
4
+ cache: bundler
5
+ rvm:
6
+ - 2.5.1
7
+ before_install: gem install bundler -v 2.0.1
data/CHANGELOG.md ADDED
@@ -0,0 +1,8 @@
1
+ # CHANGELOG
2
+
3
+ ## Next Release
4
+
5
+ 1. (New) Add initial core logic. (Issue #1)
6
+ 1. (New) Add initial CLI. (Issue #4)
7
+ 1. (New) Write results of each iteration to a separate file. (Issue #5)
8
+ 1. (New) Add `-s` option to stop on first failure. (Issue #6)
data/Gemfile ADDED
@@ -0,0 +1,8 @@
1
+ source "https://rubygems.org"
2
+
3
+ # Specify your gem's dependencies in rspec_n.gemspec
4
+ gemspec
5
+
6
+ group :development do
7
+ gem 'rubocop_plus', "~> 1.0", require: false
8
+ end
data/LICENSE.md ADDED
@@ -0,0 +1,21 @@
1
+ The MIT License (MIT)
2
+
3
+ Copyright (c) 2019 Roberts
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in
13
+ all copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21
+ THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,71 @@
1
+ # rspec_n
2
+
3
+ rspec_n is a Ruby gem that makes it easy to run a project's RSpec test suite N times. You can customize the command that is used to start RSpec, or let rspec_n guess the best command (based on the files in your project). rspec_n is useful for finding repeatability or flakiness issues in automated test suites.
4
+
5
+ ![sample](https://user-images.githubusercontent.com/2053901/52986788-d0bb7c80-33c6-11e9-9f13-0e191bdd2bb3.png)
6
+
7
+ ## Version Policy
8
+
9
+ Releases are versioned using [semver 2.0.0](https://semver.org/spec/v2.0.0.html).
10
+
11
+ ## Installation
12
+
13
+ Install by executing
14
+
15
+ $ gem install rspec_n
16
+
17
+ The gem will install an exectuable called `rspec_n` on your system. You may also want to add `rspec_n_iteration.*` to your `.gitignore` to exclude the output generated by rspec_n from your project's repo.
18
+
19
+ ## Usage
20
+
21
+ To run RSpec N times
22
+
23
+ $ rspec_n N
24
+
25
+ If your project has a `bin/start_rspec` file, rspec_n will invoke it to start RSpec (rspec_n assumes you have placed the necessary commands to cleanly start your test suite in that file). Otherwise, it will attempt to identify your project type and use best practices to start RSpec for your given project type. The following is a list of project types supported by rspec_n, and the commands used when that project type is selected:
26
+
27
+ 1. Ruby on Rails - `DISABLE_DATABASE_ENVIRONMENT_CHECK=1 RAILS_ENV=test bundle exec rake db:drop db:create db:migrate && bundle exec rspec`.
28
+
29
+ If it cannot guess a project type, rspec_n will invoke `bundle exec rspec`. You can override this by specifying a command directly on the CLI.
30
+
31
+ #### Use Custom Command to Start RSpec
32
+
33
+ By default, rspec_n will inspect the files in your project and pick the best way to start RSpec. You can take control by using the `-c` option which lets you specify a custom command. The following example deletes the `tmp` folder before starting RSpec:
34
+
35
+ $ rspec_n 5 -c 'rm -rf tmp && bundle exec rspec'
36
+
37
+ You must wrap your entire command string in single or double quotes if it contains spaces (which it probably will). This command will be invoked each time rspec_n attempts to start RSpec.
38
+
39
+ #### Control spec Order
40
+
41
+ rspec_n provides three command line options for controlling the execution order of your specs.
42
+
43
+ 1. `--order rand` - Passes `--order rand` to RSpec which causes RSpec to run your specs in a random order.
44
+ 1. `--order defined` - Passes `--order defined` to RSpec which causes RSpec to run your specs in the order that they are loaded by RSpec.
45
+ 1. `--order project` - Passes nothing to RSpec, which means the project configuration files will determine the order.
46
+
47
+ If you do nothing, rspec_n will use `--order rand`. This ensures the specs in your project are executed in random order for each iteration of RSpec. rspec_n doesn't supply a seed when it start RSpec using a random strategy; it lets RSpec choose the seed and reports the values in the results.
48
+
49
+ #### Control File Ouput
50
+
51
+ rspec_n writes output for each iteration in a sequence of files `rspec_n_iteration.1`, `rspec_n_iteration.2`, etc... This saves you from having to rerun your test suite when you find a particular seed that causes your test suite to fail. If you want to disable this, add the `--no-file` option to the command.
52
+
53
+ **Note:** rspec_n deletes all files matching `rspec_n_iteration.*` when it starts so you must move those files to another location if you want to save them.
54
+
55
+ #### Stop on First Failure
56
+
57
+ You can tell rspec_n to abort when an iteration fails by using the `-s` flag. Any remaining iterations will be skipped.
58
+
59
+ ## Development
60
+
61
+ After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for a Pry console, or `rake console` for an IRB console, that will allow you to experiment.
62
+
63
+ To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
64
+
65
+ ## Contributing
66
+
67
+ Bug reports and pull requests are welcome on GitHub at https://github.com/roberts1000/rspec_n.
68
+
69
+ ## License
70
+
71
+ The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
data/Rakefile ADDED
@@ -0,0 +1,11 @@
1
+ require "bundler/gem_tasks"
2
+ require "rspec/core/rake_task"
3
+
4
+ RSpec::Core::RakeTask.new(:spec)
5
+
6
+ task :default => :spec
7
+
8
+ task :console do
9
+ # exec "pry -r rspec_n -I ./lib"
10
+ exec "irb -r rspec_n -I ./lib"
11
+ end
data/bin/console ADDED
@@ -0,0 +1,13 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ require "bundler/setup"
4
+ require "rspec_n"
5
+
6
+ # You can add fixtures and/or initialization code here to make experimenting
7
+ # with your gem easier. You can also use a different console, if you like.
8
+
9
+ require "pry"
10
+ Pry.start
11
+
12
+ # require "irb"
13
+ # IRB.start(__FILE__)
data/bin/setup ADDED
@@ -0,0 +1,8 @@
1
+ #!/usr/bin/env bash
2
+ set -euo pipefail
3
+ IFS=$'\n\t'
4
+ set -vx
5
+
6
+ bundle install
7
+
8
+ # Do any other automated setup that you need to do here
data/exe/rspec_n ADDED
@@ -0,0 +1,74 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ # This script runs the rspec_n executable.
4
+ #
5
+ # EXECUTION
6
+ #
7
+ # rspec_n # Run the project's RSpec test suite 10 times.
8
+ # rspec_n N # Run the projects's test suite N times.
9
+ #
10
+ # EXIT STATUS
11
+ #
12
+ # See below for exit status codes and their meaning.
13
+
14
+ lib = File.expand_path('../lib', __dir__)
15
+ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
16
+
17
+ require "cri"
18
+ require "rspec_n/constants"
19
+ require "rspec_n/errors"
20
+
21
+ # rubocop:disable Metrics/BlockLength
22
+ command = Cri::Command.define do
23
+ usage 'rspec_n [iterations] [options]'
24
+ description "rspec_n is an executable installed by the rspec_n Ruby gem. It provides a " \
25
+ "way to re-run an RSpec test suite 'N' times, which is useful when determining " \
26
+ "if a test suite is consistent.\n\n" \
27
+ "STATUS CODES\n\n" \
28
+ "0 - rspec_n ran successfully \u00A0\n" \
29
+ "1 - problem with the commandline options \u00A0"
30
+
31
+ flag :h, :help, 'show help' do |_value, cmd|
32
+ puts cmd.help
33
+ exit 0
34
+ end
35
+
36
+ command_description = "By default, rspec_n will guess the best command to run RSpec; " \
37
+ "override it by setting a custom command; wrap the entire command in single or double quotes; " \
38
+ "join multiple commands with '&&'"
39
+ option :c, :command, command_description, argument: :required
40
+
41
+ flag nil, "no-file", "Do not write iteration output to files"
42
+
43
+ flag :s, "stop-fast", "Stop when an iteration reports a failure."
44
+
45
+ command_description =
46
+ "The order RSpec uses when running specs. Can be one of three values: \u00A0\n" \
47
+ "\u00A0\u00A0defined - Run specs in same order \u00A0\n" \
48
+ "\u00A0\u00A0project - Let project decide \u00A0\n" \
49
+ "\u00A0\u00A0rand - Randomize specs (default)"
50
+ option :o, :order, command_description, argument: :required
51
+
52
+ flag :v, :version, 'show the current version' do |_value, _cmd|
53
+ puts RspecN::VERSION
54
+ exit 0
55
+ end
56
+
57
+ run do |options, args, _cmd|
58
+ require "rspec_n"
59
+ RspecN::Runner.new(options, args).start
60
+ end
61
+ end
62
+ # rubocop:enable Metrics/BlockLength
63
+
64
+ begin
65
+ command.run(ARGV)
66
+ exit 0
67
+ # Raised when the user specifies an argument incorrectly.
68
+ rescue RspecN::BadArgument => e
69
+ warn e.message.colorize(:red)
70
+ exit 1
71
+ rescue RspecN::BadOption => e
72
+ warn e.message.colorize(:red)
73
+ exit 1
74
+ end
@@ -0,0 +1,9 @@
1
+ module RspecN
2
+ ALLOWED_ORDER_OPTIONS = %w[defined project rand].freeze
3
+ DEFAULT_ITERATIONS = 10
4
+ DEFAULT_COMMAND = 'bundle exec rspec'.freeze
5
+ DEFAULT_RSPEC_STARTER_COMMAND = 'bin/start_rspec'.freeze
6
+ # rubocop:disable Metrics/LineLength
7
+ DEFAULT_RAILS_COMMAND = 'DISABLE_DATABASE_ENVIRONMENT_CHECK=1 RAILS_ENV=test bundle exec rake db:drop db:create db:migrate && bundle exec rspec'.freeze
8
+ # rubocop:enable Metrics/LineLength
9
+ end
@@ -0,0 +1,18 @@
1
+ module RspecN
2
+ class BadArgument < StandardError
3
+ def initialize(msg="")
4
+ @details = msg
5
+ super
6
+ end
7
+
8
+ def message
9
+ "There was an error with the argument. rspec_n only accepts a single argument, a number greater than 0, which \n" \
10
+ "specifies how times RSpec should run. You entered:\n\n"\
11
+ " #{@details}\n\n" \
12
+ "Here are some example ways to use rspec_n (some of these may not be valid for your particular test suite):\n\n" \
13
+ " rspec_n 5\n" \
14
+ " rspec_n 3 --command 'bundle exec rspec'\n" \
15
+ " rspec_n --command 'bin/rails db:test:prepare && bundle exec rspec'"
16
+ end
17
+ end
18
+ end
@@ -0,0 +1,13 @@
1
+ module RspecN
2
+ class BadOption < StandardError
3
+ def initialize(msg="")
4
+ @details = msg
5
+ super
6
+ end
7
+
8
+ def message
9
+ allowed = RspecN::ALLOWED_ORDER_OPTIONS.collect { |val| "'" + val + "'" }
10
+ "Order must be #{allowed.to_sentence}. '#{@details}' is not allowed.\n"
11
+ end
12
+ end
13
+ end
@@ -0,0 +1,2 @@
1
+ require_relative "errors/bad_argument"
2
+ require_relative "errors/bad_option"
@@ -0,0 +1,35 @@
1
+ module RspecN
2
+ module Formatters
3
+ class FileFormatter
4
+ include RspecN::TimeHelpers
5
+
6
+ BASE_FILE_NAME = "rspec_n_iteration"
7
+
8
+ def initialize(runner:)
9
+ @runner = runner
10
+ @format = "%m/%d %l:%M:%S %p"
11
+ delete_all_files
12
+ end
13
+
14
+ def delete_all_files
15
+ Dir.glob("#{BASE_FILE_NAME}.**").each { |file| File.delete(file)}
16
+ end
17
+
18
+ def write(run)
19
+ return if run.skipped?
20
+ return unless @runner.input.write_files?
21
+
22
+ file_name = "#{BASE_FILE_NAME}.#{run.iteration}"
23
+
24
+ File.open(file_name, "w") do |f|
25
+ f.write("Iteration: #{run.iteration}\n")
26
+ f.write("Start Time: #{run.formatted_start_time(@format)}\n")
27
+ f.write("Finish Time: #{run.formatted_finish_time(@format)}\n")
28
+ f.write("Duration: #{convert_seconds_to_hms(run.duration_seconds)}\n\n")
29
+ f.write(run.rspec_stdout)
30
+ f.write(run.rspec_stderr)
31
+ end
32
+ end
33
+ end
34
+ end
35
+ end
@@ -0,0 +1,93 @@
1
+ module RspecN
2
+ module Formatters
3
+ class TableFormatter
4
+ include RspecN::TimeHelpers
5
+
6
+ attr_accessor :columns
7
+
8
+ def initialize(runner:)
9
+ @runner = runner
10
+ @columns = { "Run" => 8, "Start Time" => 22, "Finish Time" => 22, "Duration" => 13, "Seed" => 11, "Result" => 18 }
11
+ @header_columns_string = padded_header_column_labels
12
+ @table_width = @header_columns_string.size
13
+ @format = "%m/%d %l:%M:%S %p"
14
+ end
15
+
16
+ def observe
17
+ write_table_header
18
+ yield
19
+ write_conclusion
20
+ end
21
+
22
+ def show_pre_run_info(run)
23
+ print pad_field("Run", run.iteration)
24
+ print pad_field("Start Time", run.formatted_start_time(@format))
25
+ end
26
+
27
+ def show_post_run_info(run)
28
+ print pad_field("Finish Time", run.formatted_finish_time(@format))
29
+ print duration_field(run)
30
+ print seed_field(run)
31
+ puts result_field(run)
32
+ end
33
+
34
+ private
35
+
36
+ def write_table_header
37
+ puts @header_columns_string
38
+ puts "-" * @table_width
39
+ end
40
+
41
+ def write_conclusion
42
+ puts "-" * @table_width
43
+ puts ""
44
+ puts "Total Duration: #{convert_seconds_to_hms(@runner.total_duration_seconds)}"
45
+ puts "Average Duration: #{convert_seconds_to_hms(@runner.avg_duration_seconds)}"
46
+ puts "Total Passed: #{@runner.total_passed.to_s.colorize(:green)}"
47
+ puts "Total Passed with Warnings: #{@runner.total_passed_with_warnings.to_s.colorize(:green)}"
48
+ puts "Total Failed: #{@runner.total_failed.to_s.colorize(:red)}"
49
+ puts "Total Skipped: #{@runner.total_skipped}"
50
+ puts ""
51
+ end
52
+
53
+ def padded_header_column_label(name, width)
54
+ right_padding = width - name.size
55
+ name + (" " * right_padding)
56
+ end
57
+
58
+ def padded_header_column_labels
59
+ columns.collect { |name, max_width| padded_header_column_label(name, max_width) }.join("")
60
+ end
61
+
62
+ def max_column_width_for(name)
63
+ columns[name]
64
+ end
65
+
66
+ def pad_field(column_name, value)
67
+ max_width = max_column_width_for(column_name)
68
+ value_size = value.to_s.size
69
+ pad_count = max_width - value_size
70
+ value.to_s + (" " * pad_count)
71
+ end
72
+
73
+ def duration_field(run)
74
+ hms = convert_seconds_to_hms(run.duration_seconds)
75
+ pad_field("Duration", hms)
76
+ end
77
+
78
+ def seed_field(run)
79
+ run.seed.nil? ? pad_field("Seed", "None") : pad_field("Seed", run.seed)
80
+ end
81
+
82
+ def result_field(run)
83
+ case run.status_string
84
+ when "Pass with Warnings" then "Pass with Warnings".colorize(:yellow)
85
+ when "Pass" then "Pass".colorize(:green)
86
+ when "Fail" then "Fail".colorize(:red)
87
+ when "Skip" then "Skip".colorize(:yellow)
88
+ else "Unknown".colorize(:yellow)
89
+ end
90
+ end
91
+ end
92
+ end
93
+ end
@@ -0,0 +1,14 @@
1
+ class Array
2
+ def to_sentence(word_connector: ", ", two_word_connector: " or ", last_word_connector: " or ")
3
+ case length
4
+ when 0
5
+ ""
6
+ when 1
7
+ self[0].to_s.dup
8
+ when 2
9
+ "#{self[0]}#{two_word_connector}#{self[1]}"
10
+ else
11
+ "#{self[0...-1].join(word_connector)}#{last_word_connector}#{self[-1]}"
12
+ end
13
+ end
14
+ end
@@ -0,0 +1,6 @@
1
+ class String
2
+ # https://stackoverflow.com/questions/39743299/check-if-a-string-contains-only-digits-in-ruby
3
+ def all_digits?
4
+ self =~ /\A\d+\z/ ? true : false
5
+ end
6
+ end
@@ -0,0 +1,14 @@
1
+ module RspecN
2
+ module TimeHelpers
3
+ def convert_seconds_to_hms(total_seconds)
4
+ hours = total_seconds / (60 * 60)
5
+ minutes = (total_seconds / 60) % 60
6
+ seconds = total_seconds % 60
7
+ [hours, minutes, seconds].map do |t|
8
+ # Right justify and pad with 0 until length is 2.
9
+ # So if the duration of any of the time components is 0, then it will display as 00
10
+ t.round.to_s.rjust(2, '0')
11
+ end.join(':')
12
+ end
13
+ end
14
+ end
@@ -0,0 +1,61 @@
1
+ module RspecN
2
+ class Input
3
+ attr_accessor :iterations, :command, :stop_fast, :write_files
4
+ def initialize(options, args)
5
+ @args = args
6
+ @options = options
7
+ validate_args
8
+ validate_order
9
+ @iterations = determine_iterations
10
+ @order = options.fetch(:order, "rand")
11
+ @command = determine_command
12
+ @stop_fast = options.fetch(:"stop-fast", false)
13
+ @write_files = !options.fetch(:'no-file', false)
14
+ end
15
+
16
+ def write_files?
17
+ @write_files
18
+ end
19
+
20
+ private
21
+
22
+ def validate_args
23
+ return if @args.size.zero?
24
+ raise BadArgument, @args.join(', ') if @args.empty? || !@args.first.all_digits?
25
+ raise BadArgument, @args.first if @args.first.to_i < 1
26
+ end
27
+
28
+ def validate_order
29
+ return unless (order = @options.fetch(:order, nil))
30
+
31
+ raise BadOption, order unless RspecN::ALLOWED_ORDER_OPTIONS.include?(order)
32
+ end
33
+
34
+ def determine_iterations
35
+ @args.empty? ? RspecN::DEFAULT_ITERATIONS : @args.first.to_i
36
+ end
37
+
38
+ def determine_command
39
+ return @options[:command] if @options.fetch(:command, nil)
40
+
41
+ guessed_command + " --order " + @order
42
+ end
43
+
44
+ def guessed_command
45
+ return RspecN::DEFAULT_RSPEC_STARTER_COMMAND if project_uses_rspec_starter?
46
+ return RspecN::DEFAULT_RAILS_COMMAND if project_is_rails_based?
47
+
48
+ RspecN::DEFAULT_COMMAND
49
+ end
50
+
51
+ def project_uses_rspec_starter?
52
+ app_file_name = "bin/start_rspec"
53
+ File.file?(app_file_name)
54
+ end
55
+
56
+ def project_is_rails_based?
57
+ app_file_name = "config/application.rb"
58
+ File.file?(app_file_name) && File.readlines(app_file_name).grep(/Rails::Application/).any?
59
+ end
60
+ end
61
+ end
@@ -0,0 +1,83 @@
1
+ module RspecN
2
+ class Run
3
+ attr_accessor :iteration, :start_time, :finish_time, :seed, :rspec_stdout, :rspec_stderr, :rspec_status, :duration_seconds, :status_string
4
+
5
+ def initialize(iteration:)
6
+ @iteration = iteration
7
+ @start_time = nil
8
+ @finish_time = nil
9
+ @duration_seconds = nil
10
+ @seed = nil
11
+ @rspec_stdout = nil
12
+ @rspec_stderr = nil
13
+ @rspec_status = nil
14
+ @status_string = nil
15
+ @skipped = false
16
+ end
17
+
18
+ def start_clock
19
+ @start_time = Time.now
20
+ end
21
+
22
+ def stop_clock
23
+ @finish_time = Time.now
24
+ finalize_duration_seconds
25
+ finalize_seed
26
+ finalize_status_string
27
+ end
28
+
29
+ def go(command)
30
+ @rspec_stdout, @rspec_stderr, @rspec_status = Open3.capture3(command)
31
+ end
32
+
33
+ def formatted_start_time(format)
34
+ start_time.strftime(format)
35
+ end
36
+
37
+ def formatted_finish_time(format)
38
+ finish_time.strftime(format)
39
+ end
40
+
41
+ def passed?
42
+ @status_string == "Pass"
43
+ end
44
+
45
+ def passed_with_warnings?
46
+ @status_string == "Pass with Warnings"
47
+ end
48
+
49
+ def skip
50
+ @skipped = true
51
+ @duration_seconds = 0
52
+ end
53
+
54
+ def skipped?
55
+ @skipped
56
+ end
57
+
58
+ def failed?
59
+ @status_string == "Fail"
60
+ end
61
+
62
+ private
63
+
64
+ def finalize_duration_seconds
65
+ @duration_seconds = (@finish_time - @start_time).round
66
+ end
67
+
68
+ def finalize_seed
69
+ result = @rspec_stdout.match(/^Randomized with seed (\d*)/)
70
+ return if result.nil? # A seed wasn't used
71
+
72
+ @seed = result.captures.first&.strip
73
+ end
74
+
75
+ def finalize_status_string
76
+ return @status_string = "Skip" if skipped?
77
+ return @status_string = "Pass with Warnings" if @rspec_status.exitstatus.zero? && !@rspec_stderr.empty?
78
+ return @status_string = "Pass" if @rspec_status.exitstatus.zero?
79
+ return @status_string = "Fail" if !@rspec_status.exitstatus.zero?
80
+ @status_string = "Undetermined"
81
+ end
82
+ end
83
+ end
@@ -0,0 +1,71 @@
1
+ module RspecN
2
+ class Runner < Cri::CommandRunner
3
+ attr_reader :command, :input, :iterations, :runs
4
+
5
+ def initialize(options, args)
6
+ @input = Input.new(options, args)
7
+ @iterations = @input.iterations
8
+ @command = @input.command
9
+ @display_formatter = Formatters::TableFormatter.new(runner: self)
10
+ @file_formatter = Formatters::FileFormatter.new(runner: self)
11
+ initialize_runs
12
+ end
13
+
14
+ def start
15
+ display_intro
16
+ @display_formatter.observe { run_tests }
17
+ end
18
+
19
+ def total_duration_seconds
20
+ @runs.values.inject(0) { |sum, run| sum + run.duration_seconds }
21
+ end
22
+
23
+ def avg_duration_seconds
24
+ (total_duration_seconds / @iterations).floor
25
+ end
26
+
27
+ def total_passed
28
+ @runs.values.select { |run| run.passed? }.size
29
+ end
30
+
31
+ def total_passed_with_warnings
32
+ @runs.values.select { |run| run.passed_with_warnings? }.size
33
+ end
34
+
35
+ def total_failed
36
+ @runs.values.select { |run| run.failed? }.size
37
+ end
38
+
39
+ def total_skipped
40
+ @runs.values.select { |run| run.skipped? }.size
41
+ end
42
+
43
+ private
44
+
45
+ def initialize_runs
46
+ @runs = {}
47
+ @iterations.times { |i| @runs[i + 1] = Run.new(iteration: i + 1) }
48
+ end
49
+
50
+ def run_tests
51
+ found_failure = false
52
+
53
+ @runs.each do |_iteration, run|
54
+ next run.skip if @input.stop_fast && found_failure
55
+
56
+ run.start_clock
57
+ @display_formatter.show_pre_run_info(run)
58
+ run.go(@command)
59
+ run.stop_clock
60
+ @display_formatter.show_post_run_info(run)
61
+ found_failure ||= run.failed?
62
+ @file_formatter.write(run)
63
+ end
64
+ end
65
+
66
+ def display_intro
67
+ iteration_part = @iterations > 1 ? "#{@iterations} times" : "1 time"
68
+ puts "\nRSpec will execute #{iteration_part.colorize(:yellow)} using #{command.to_s.colorize(:yellow)}\n\n"
69
+ end
70
+ end
71
+ end
@@ -0,0 +1,3 @@
1
+ module RspecN
2
+ VERSION = "1.0.0".freeze
3
+ end
data/lib/rspec_n.rb ADDED
@@ -0,0 +1,28 @@
1
+ require "rspec_n/version"
2
+
3
+ require "colorize"
4
+ require "cri"
5
+ require "open3"
6
+
7
+ require "rspec_n/helpers/time_helpers"
8
+ require "rspec_n/helpers/core_ext/array"
9
+ require "rspec_n/helpers/core_ext/string"
10
+
11
+ require "rspec_n/errors"
12
+
13
+ require "rspec_n/constants"
14
+ require "rspec_n/input"
15
+ require "rspec_n/runner"
16
+ require "rspec_n/run"
17
+ require "rspec_n/formatters/file_formatter"
18
+ require "rspec_n/formatters/table_formatter"
19
+
20
+ # Setup pry for development when running "rake console". Guard against load
21
+ # errors in production (since pry is only loaded as a DEVELOPMENT dependency
22
+ # in the .gemspec)
23
+ # rubocop:disable Lint/HandleExceptions
24
+ begin
25
+ require "pry"
26
+ rescue LoadError
27
+ end
28
+ # rubocop:enable Lint/HandleExceptions
data/rspec_n.gemspec ADDED
@@ -0,0 +1,45 @@
1
+ lib = File.expand_path("lib", __dir__)
2
+ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
3
+ require "rspec_n/version"
4
+
5
+ # rubocop:disable Metrics/BlockLength
6
+ Gem::Specification.new do |spec|
7
+ spec.name = "rspec_n"
8
+ spec.version = RspecN::VERSION
9
+ spec.authors = ["roberts1000"]
10
+ spec.email = ["roberts@corlewsolutions.com"]
11
+
12
+ spec.summary = "A ruby gem that runs RSpec N times."
13
+ spec.description = "A ruby gem that runs RSpec N times."
14
+ spec.homepage = "https://github.com/roberts1000/rspec_n"
15
+ spec.license = "MIT"
16
+
17
+ # Prevent pushing this gem to RubyGems.org. To allow pushes either set the 'allowed_push_host'
18
+ # to allow pushing to a single host or delete this section to allow pushing to any host.
19
+ if spec.respond_to?(:metadata)
20
+ spec.metadata["homepage_uri"] = spec.homepage
21
+ spec.metadata["source_code_uri"] = "https://github.com/roberts1000/rspec_n"
22
+ spec.metadata["changelog_uri"] = "https://github.com/roberts1000/rspec_n/blob/master/CHANGELOG.md"
23
+ else
24
+ raise "RubyGems 2.0 or newer is required to protect against " \
25
+ "public gem pushes."
26
+ end
27
+
28
+ # Specify which files should be added to the gem when it is released.
29
+ # The `git ls-files -z` loads the files in the RubyGem that have been added into git.
30
+ spec.files = Dir.chdir(File.expand_path(__dir__)) do
31
+ `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
32
+ end
33
+ spec.bindir = "exe"
34
+ spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
35
+ spec.require_paths = ["lib"]
36
+
37
+ spec.add_development_dependency "bundler", "~> 2.0"
38
+ spec.add_development_dependency "pry", "~> 0.11.2"
39
+ spec.add_development_dependency "rake", "~> 12.0"
40
+ spec.add_development_dependency "rspec", "~> 3.0"
41
+
42
+ spec.add_dependency "colorize", "~> 0.8.0"
43
+ spec.add_dependency "cri", "~> 2.15.3"
44
+ end
45
+ # rubocop:enable Metrics/BlockLength
metadata ADDED
@@ -0,0 +1,159 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: rspec_n
3
+ version: !ruby/object:Gem::Version
4
+ version: 1.0.0
5
+ platform: ruby
6
+ authors:
7
+ - roberts1000
8
+ autorequire:
9
+ bindir: exe
10
+ cert_chain: []
11
+ date: 2019-02-19 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: bundler
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - "~>"
18
+ - !ruby/object:Gem::Version
19
+ version: '2.0'
20
+ type: :development
21
+ prerelease: false
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - "~>"
25
+ - !ruby/object:Gem::Version
26
+ version: '2.0'
27
+ - !ruby/object:Gem::Dependency
28
+ name: pry
29
+ requirement: !ruby/object:Gem::Requirement
30
+ requirements:
31
+ - - "~>"
32
+ - !ruby/object:Gem::Version
33
+ version: 0.11.2
34
+ type: :development
35
+ prerelease: false
36
+ version_requirements: !ruby/object:Gem::Requirement
37
+ requirements:
38
+ - - "~>"
39
+ - !ruby/object:Gem::Version
40
+ version: 0.11.2
41
+ - !ruby/object:Gem::Dependency
42
+ name: rake
43
+ requirement: !ruby/object:Gem::Requirement
44
+ requirements:
45
+ - - "~>"
46
+ - !ruby/object:Gem::Version
47
+ version: '12.0'
48
+ type: :development
49
+ prerelease: false
50
+ version_requirements: !ruby/object:Gem::Requirement
51
+ requirements:
52
+ - - "~>"
53
+ - !ruby/object:Gem::Version
54
+ version: '12.0'
55
+ - !ruby/object:Gem::Dependency
56
+ name: rspec
57
+ requirement: !ruby/object:Gem::Requirement
58
+ requirements:
59
+ - - "~>"
60
+ - !ruby/object:Gem::Version
61
+ version: '3.0'
62
+ type: :development
63
+ prerelease: false
64
+ version_requirements: !ruby/object:Gem::Requirement
65
+ requirements:
66
+ - - "~>"
67
+ - !ruby/object:Gem::Version
68
+ version: '3.0'
69
+ - !ruby/object:Gem::Dependency
70
+ name: colorize
71
+ requirement: !ruby/object:Gem::Requirement
72
+ requirements:
73
+ - - "~>"
74
+ - !ruby/object:Gem::Version
75
+ version: 0.8.0
76
+ type: :runtime
77
+ prerelease: false
78
+ version_requirements: !ruby/object:Gem::Requirement
79
+ requirements:
80
+ - - "~>"
81
+ - !ruby/object:Gem::Version
82
+ version: 0.8.0
83
+ - !ruby/object:Gem::Dependency
84
+ name: cri
85
+ requirement: !ruby/object:Gem::Requirement
86
+ requirements:
87
+ - - "~>"
88
+ - !ruby/object:Gem::Version
89
+ version: 2.15.3
90
+ type: :runtime
91
+ prerelease: false
92
+ version_requirements: !ruby/object:Gem::Requirement
93
+ requirements:
94
+ - - "~>"
95
+ - !ruby/object:Gem::Version
96
+ version: 2.15.3
97
+ description: A ruby gem that runs RSpec N times.
98
+ email:
99
+ - roberts@corlewsolutions.com
100
+ executables:
101
+ - rspec_n
102
+ extensions: []
103
+ extra_rdoc_files: []
104
+ files:
105
+ - ".gitignore"
106
+ - ".rspec"
107
+ - ".rubocop.yml"
108
+ - ".travis.yml"
109
+ - CHANGELOG.md
110
+ - Gemfile
111
+ - LICENSE.md
112
+ - README.md
113
+ - Rakefile
114
+ - bin/console
115
+ - bin/setup
116
+ - exe/rspec_n
117
+ - lib/rspec_n.rb
118
+ - lib/rspec_n/constants.rb
119
+ - lib/rspec_n/errors.rb
120
+ - lib/rspec_n/errors/bad_argument.rb
121
+ - lib/rspec_n/errors/bad_option.rb
122
+ - lib/rspec_n/formatters/file_formatter.rb
123
+ - lib/rspec_n/formatters/table_formatter.rb
124
+ - lib/rspec_n/helpers/core_ext/array.rb
125
+ - lib/rspec_n/helpers/core_ext/string.rb
126
+ - lib/rspec_n/helpers/time_helpers.rb
127
+ - lib/rspec_n/input.rb
128
+ - lib/rspec_n/run.rb
129
+ - lib/rspec_n/runner.rb
130
+ - lib/rspec_n/version.rb
131
+ - rspec_n.gemspec
132
+ homepage: https://github.com/roberts1000/rspec_n
133
+ licenses:
134
+ - MIT
135
+ metadata:
136
+ homepage_uri: https://github.com/roberts1000/rspec_n
137
+ source_code_uri: https://github.com/roberts1000/rspec_n
138
+ changelog_uri: https://github.com/roberts1000/rspec_n/blob/master/CHANGELOG.md
139
+ post_install_message:
140
+ rdoc_options: []
141
+ require_paths:
142
+ - lib
143
+ required_ruby_version: !ruby/object:Gem::Requirement
144
+ requirements:
145
+ - - ">="
146
+ - !ruby/object:Gem::Version
147
+ version: '0'
148
+ required_rubygems_version: !ruby/object:Gem::Requirement
149
+ requirements:
150
+ - - ">="
151
+ - !ruby/object:Gem::Version
152
+ version: '0'
153
+ requirements: []
154
+ rubyforge_project:
155
+ rubygems_version: 2.7.8
156
+ signing_key:
157
+ specification_version: 4
158
+ summary: A ruby gem that runs RSpec N times.
159
+ test_files: []