spinach 0.10.1 → 0.11.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: 1f75603856e6e68fb9f7eb16a287062659034e02
-  data.tar.gz: a6ea70aaa3ef522f10e6b5c729671c94c84bc67c
+SHA256:
+  metadata.gz: 66296e07e2af7b9daa268623eb0e59ac1e90ee16855a3baf9eb63801be1338d2
+  data.tar.gz: b2239f90a08a88a7b9c0ca93d16103b816249122c60176ea094405f8d1f8ebf8
 SHA512:
-  metadata.gz: c5f580a7a0bc2c9f4ce8901b78f1f63da2a72d14f8233ade3ebd423f6bf0989441b318970266610422b6e53de854fd6cf1ce868f53e80064f6606396d9c39c3f
-  data.tar.gz: 349c9aa0672871576ea8ff8f86944c24816048ccbee2ca3a360b446feed8d6ede0ab2671a8db2505ea4cfd80a0f3e5e66f55578e88d69db215981d37970c06a3
+  metadata.gz: 909a2ec0656f8991d9ab05a3a26864544da914475de5d8205ada4a20b3041c8cddb81d33319329252a440ec53e32852ff2daf0a0364fe9d87b2c1bff815725c4
+  data.tar.gz: b08372a0eb16eb49f0ca4972f8a412307ea9cd0eba8eab48c848ff849700bad2dc79444f6eb73d3830ed67941c0ce3efa44803658aa71da453a10fa00fa114c1

data/.github/workflows/ci.yml

@@ -0,0 +1,31 @@
+name: "Tests"
+
+on:
+  push:
+    branches:
+      - master
+  pull_request:
+    branches:
+      - "*"
+
+env:
+  CI: "true"
+
+jobs:
+  main:
+    name: Tests
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        ruby: [2.4, 2.5, 2.6, 2.7, jruby]
+      fail-fast: false
+    steps:
+      - uses: actions/checkout@v2.0.0
+        with:
+          fetch-depth: 1
+      - uses: ruby/setup-ruby@master
+        with:
+          ruby-version: ${{ matrix.ruby }}
+      - run: bundle install --jobs 4 --retry 3
+        name: Install Ruby deps
+      - run: bundle exec rake

data/.ruby-version

@@ -1 +1 @@
-2.3.1
+2.7.1

data/README.markdown

@@ -1,8 +1,9 @@
 # Spinach - BDD framework on top of Gherkin
-[![Gem Version](https://badge.fury.io/rb/spinach.png)](http://badge.fury.io/rb/spinach)
-[![Build Status](https://secure.travis-ci.org/codegram/spinach.png)](http://travis-ci.org/codegram/spinach)
-[![Dependency Status](https://gemnasium.com/codegram/spinach.png)](http://gemnasium.com/codegram/spinach)
+
+[![Gem Version](https://badge.fury.io/rb/spinach.svg)](https://badge.fury.io/rb/spinach)
+![Tests](https://github.com/codegram/spinach/workflows/Tests/badge.svg)
 [![Coverage Status](https://coveralls.io/repos/codegram/spinach/badge.png?branch=master)](https://coveralls.io/r/codegram/spinach)
+![](https://ruby-gem-downloads-badge.herokuapp.com/rails)
 
 Spinach is a high-level BDD framework that leverages the expressive
 [Gherkin language][gherkin] (used by [Cucumber][cucumber]) to help you define
@@ -10,13 +11,13 @@ executable specifications of your application or library's acceptance criteria.
 
 Conceived as an alternative to Cucumber, here are some of its design goals:
 
-* Step maintainability: since features map to their own classes, their steps are
+- Step maintainability: since features map to their own classes, their steps are
   just methods of that class. This encourages step encapsulation.
 
-* Step reusability: In case you want to reuse steps across features, you can
+- Step reusability: In case you want to reuse steps across features, you can
   always wrap those in plain ol' Ruby modules.
 
-Spinach is tested against **2.0, 2.1, 2.2 and 2.3** as well as **JRuby 9000**.
+Spinach is tested against Ruby MRI **2.4, 2.5, 2.6 and 2.7** as well as **latest JRuby 9000**.
 
 ## Getting started
 
@@ -184,7 +185,7 @@ end
 ## Audit
 
 Over time, the definitions of your features will change. When you add, remove
-or change steps in the feature files, you can easily audit your existing step 
+or change steps in the feature files, you can easily audit your existing step
 files with:
 
 ```shell
@@ -198,12 +199,11 @@ This does not modify the step files, so you will need to paste the boilerplate
 into the appropriate places. If a new feature file is detected, you will be
 asked to run `spinach --generate` beforehand.
 
-**Important**: If auditing individual files, common steps (as above) may be 
-reported as unused when they are actually used in a feature file that is not 
+**Important**: If auditing individual files, common steps (as above) may be
+reported as unused when they are actually used in a feature file that is not
 currently being audited. To avoid this, run the audit with no arguments to
 audit all step files simultaneously.
 
-
 ## Tags
 
 Feature and Scenarios can be marked with tags in the form: `@tag`. Tags can be
@@ -238,19 +238,19 @@ Feature: So something great
   Scenario: Ensure no regression on this
 ```
 
-  Then you can run all Scenarios in your suite related to `@feat-1` using:
+Then you can run all Scenarios in your suite related to `@feat-1` using:
 
 ```shell
 $ spinach --tags @feat-1
 ```
 
-  Or only Scenarios related to `@feat-1` and `@bug-12` using:
+Or only Scenarios related to `@feat-1` and `@bug-12` using:
 
 ```shell
 $ spinach --tags @feat-1,@bug-12
 ```
 
-  Or only Scenarios related to `@feat-1` excluding `@bug-12` using:
+Or only Scenarios related to `@feat-1` excluding `@bug-12` using:
 
 ```shell
 $ spinach --tags @feat-1,~@bug-12
@@ -292,7 +292,7 @@ Full hook documentation is here:
 
 ## Local Before and After Hooks
 
-Sometimes it feels awkward to add steps into feature file just because you need to do some test setup and cleanup. And it is equally awkward to add a global hooks for this purpose. For example, if you want to add a session timeout feature, to do so, you want to set the session timeout time to 1 second just for this feature, and put the normal timeout back after this feature. It doesn't make sense to add two steps in the feature file just to change the session timeout value. In this scenario, a ```before``` and ```after``` blocks are perfect for this kind of tasks. Below is an example implementation:
+Sometimes it feels awkward to add steps into feature file just because you need to do some test setup and cleanup. And it is equally awkward to add a global hooks for this purpose. For example, if you want to add a session timeout feature, to do so, you want to set the session timeout time to 1 second just for this feature, and put the normal timeout back after this feature. It doesn't make sense to add two steps in the feature file just to change the session timeout value. In this scenario, a `before` and `after` blocks are perfect for this kind of tasks. Below is an example implementation:
 
 ```ruby
 class Spinach::Features::SessionTimeout < Spinach::FeatureSteps
@@ -329,8 +329,8 @@ When no reporter is specified, `stdout` will be used by default.
 
 Other reporters:
 
-* For a console reporter with no colors, try [spinach-console-reporter][spinach-console-reporter] (to be used with Jenkins)
-* For a rerun reporter, try [spinach-rerun-reporter][spinach-rerun-reporter] (writes failed scenarios in a file)
+- For a console reporter with no colors, try [spinach-console-reporter][spinach-console-reporter] (to be used with Jenkins)
+- For a rerun reporter, try [spinach-rerun-reporter][spinach-rerun-reporter] (writes failed scenarios in a file)
 
 ## Wanna use it with Rails 3?
 
@@ -342,47 +342,46 @@ Check out our [spinach-sinatra demo](https://github.com/codegram/spinach-sinatra
 
 ## Resources
 
-* [Landing page](http://codegram.github.com/spinach)
-* [Slides](http://codegram.github.com/spinach-presentation)
-* [Blog post](http://blog.codegram.com/2011/10/how-to-achieve-more-clean-encapsulated-modular-step-definitions-with-spinach)
-* [API Documentation](http://rubydoc.info/github/codegram/spinach/master/frames)
-* [Google group](https://groups.google.com/forum/#!forum/spinach_bdd)
+- [Landing page](http://codegram.github.com/spinach)
+- [Slides](http://codegram.github.com/spinach-presentation)
+- [Blog post](http://blog.codegram.com/2011/10/how-to-achieve-more-clean-encapsulated-modular-step-definitions-with-spinach)
+- [API Documentation](http://rubydoc.info/github/codegram/spinach/master/frames)
+- [Google group](https://groups.google.com/forum/#!forum/spinach_bdd)
 
 ### Related gems
 
-* [guard-spinach](http://github.com/codegram/guard-spinach)
-* [spinach-rails](http://github.com/codegram/spinach-rails)
-* [spinach-console-reporter][spinach-console-reporter] (to be used with Jenkins)
-* [spinach-rerun-reporter][spinach-rerun-reporter] (writes failed scenarios in a file)
-* [spring-commands-spinach](https://github.com/jvanbaarsen/spring-commands-spinach) (to be used with spring)
+- [guard-spinach](http://github.com/codegram/guard-spinach)
+- [spinach-rails](http://github.com/codegram/spinach-rails)
+- [spinach-console-reporter][spinach-console-reporter] (to be used with Jenkins)
+- [spinach-rerun-reporter][spinach-rerun-reporter] (writes failed scenarios in a file)
+- [spring-commands-spinach](https://github.com/jvanbaarsen/spring-commands-spinach) (to be used with spring)
 
 ### Demos
 
-* [spinach rails demo](https://github.com/codegram/spinach-rails-demo)
-* [spinach sinatra demo](https://github.com/codegram/spinach-sinatra-demo)
-* [simple todo Rails app](https://github.com/codegram/tasca-spinach-demo)
+- [spinach rails demo](https://github.com/codegram/spinach-rails-demo)
+- [spinach sinatra demo](https://github.com/codegram/spinach-sinatra-demo)
+- [simple todo Rails app](https://github.com/codegram/tasca-spinach-demo)
 
 ## Contributing
 
-* [List of spinach contributors](https://github.com/codegram/spinach/contributors)
+- [List of spinach contributors](https://github.com/codegram/spinach/contributors)
 
 You can easily contribute to Spinach. Its codebase is simple and
 [extensively documented][documentation].
 
-* Fork the project.
-* Make your feature addition or bug fix.
-* Add specs for it. This is important so we don't break it in a future
+- Fork the project.
+- Make your feature addition or bug fix.
+- Add specs for it. This is important so we don't break it in a future
   version unintentionally.
-* Commit, do not mess with rakefile, version, or history.
+- Commit, do not mess with rakefile, version, or history.
   If you want to have your own version, that is fine but bump version
   in a commit by itself I can ignore when I pull.
-* Send me a pull request. Bonus points for topic branches.
+- Send me a pull request. Bonus points for topic branches.
 
 ## License
 
 MIT (Expat) License. Copyright 2011-2016 [Codegram Technologies](http://codegram.com)
 
-
 [gherkin]: http://github.com/codegram/gherkin-ruby
 [cucumber]: http://github.com/cucumber/cucumber
 [documentation]: http://rubydoc.info/github/codegram/spinach/master/frames

data/Rakefile

@@ -11,7 +11,7 @@ end
 
 desc 'Run spinach features'
 task :spinach do
-  exec "bin/spinach"
+  exec "bin/spinach --rand"
 end
 
 task :default => [:test, :spinach]

data/features/randomization.feature

@@ -0,0 +1,16 @@
+Feature: Randomizing Features & Scenarios
+  In order to ensure my tests aren't dependent
+  As a developer
+  I want spinach to randomize features and scenarios (but not steps)
+
+  Scenario: Randomizing the run without specifying a seed
+    Given I have 2 features with 2 scenarios each
+    When I randomize the run without specifying a seed
+    Then The features and scenarios are run
+    And The runner output shows a seed
+
+  Scenario: Specifying the seed
+    Given I have 2 features with 2 scenarios each
+    When I specify the seed for the run
+    Then The features and scenarios are run in a different order
+    And The runner output shows the seed

data/features/reporting/display_run_summary.feature

@@ -1,9 +1,25 @@
 Feature: Display run summary
   As a developer
   I want spinach to display a summary of steps statuses
-  So I can easyly know general features status
+  So I can easily know general features status
 
-  Scenario: Display run summary at the end of features run
+  Scenario: Display run summary at the end of features run without randomization
     Given I have a feature that has some successful, undefined, failed and error steps
-    When I run it
+
+    When I run it without randomization
+    Then I should see a summary with steps status information
+    And I shouldn't see a randomization seed
+
+  Scenario: Display run summary at the end of features run with randomization
+    Given I have a feature that has some successful, undefined, failed and error steps
+
+    When I run it with randomization
+    Then I should see a summary with steps status information
+    And I should see a randomization seed
+
+  Scenario: Display run summary at the end of features run with a randomization seed
+    Given I have a feature that has some successful, undefined, failed and error steps
+
+    When I run it with a specific randomization seed
     Then I should see a summary with steps status information
+    And I should see that specific randomization seed

data/features/steps/randomizing_features_scenarios.rb

@@ -0,0 +1,68 @@
+class Spinach::Features::RandomizingFeaturesScenarios < Spinach::FeatureSteps
+  include Integration::SpinachRunner
+
+  step 'I have 2 features with 2 scenarios each' do
+    write_file 'features/success_a.feature', <<-FEATURE
+Feature: Success A
+  Scenario: A1
+    Then a1
+  Scenario: A2
+    Then a2
+    FEATURE
+
+    write_file 'features/steps/success_a.rb', <<-STEPS
+class Spinach::Features::SuccessA < Spinach::FeatureSteps
+  step 'a1' do; end
+  step 'a2' do; end
+end
+    STEPS
+
+    write_file 'features/success_b.feature', <<-FEATURE
+Feature: Success B
+  Scenario: B1
+    Then b1
+  Scenario: B2
+    Then b2
+    FEATURE
+
+    write_file 'features/steps/success_b.rb', <<-STEPS
+class Spinach::Features::SuccessB < Spinach::FeatureSteps
+  step 'b1' do; end
+  step 'b2' do; end
+end
+    STEPS
+  end
+
+  step 'I randomize the run without specifying a seed' do
+    run_spinach({append: "--rand"})
+  end
+
+  step 'I specify the seed for the run' do
+    # Reverse order (A2 A1 B2 B1) is the only way I can show that
+    # scenarios and features are randomized by the seed when the
+    # example has 2 features each with 2 scenarios. I tried seeds
+    # until I found one that ordered the test in that order.
+    @seed = 1
+
+    run_spinach({append: "--seed #{@seed}"})
+  end
+
+  step 'The features and scenarios are run' do
+    @stdout.must_include("A1")
+    @stdout.must_include("A2")
+    @stdout.must_include("B1")
+    @stdout.must_include("B2")
+  end
+
+  step 'The features and scenarios are run in a different order' do
+    @stdout.must_match(/B2.*B1.*A2.*A1/m)
+  end
+
+  step 'The runner output shows a seed' do
+    @stdout.must_match(/^Randomized with seed \d*$/)
+  end
+
+  step 'The runner output shows the seed' do
+    @stdout.must_match(/^Randomized with seed #{@seed}$/)
+  end
+end

data/features/steps/reporting/display_run_summary.rb

@@ -62,7 +62,7 @@ Feature: A test feature
     @feature = "features/test_feature.feature"
   end
 
-  When "I run it" do
+  When "I run it without randomization" do
     run_feature @feature
   end
 
@@ -71,4 +71,32 @@ Feature: A test feature
       /Summary:.*4.*Successful.*1.*Undefined.*1.*Failed.*1.*Error/
     )
   end
+
+  And "I shouldn't see a randomization seed" do
+    @stdout.wont_match(
+      /Randomized\ with\ seed\ \d+/
+    )
+  end
+
+  When "I run it with randomization" do
+    run_feature @feature, {append: "--rand"}
+  end
+
+  And "I should see a randomization seed" do
+    @stdout.must_match(
+      /Randomized\ with\ seed\ \d+/
+    )
+  end
+
+  When "I run it with a specific randomization seed" do
+    @seed = rand(0xFFFF)
+
+    run_feature @feature, {append: "--seed #{@seed}"}
+  end
+
+  And "I should see that specific randomization seed" do
+    @stdout.must_match(
+      /Randomized\ with\ seed\ #{@seed}/
+    )
+  end
 end

data/features/support/spinach_runner.rb

@@ -23,6 +23,17 @@ module Integration
       run "#{ruby} #{spinach} #{feature} #{options[:append]}", options[:env]
     end
 
+    def run_spinach(options = {})
+      options[:framework] ||= :minitest
+
+      use_minitest if options[:framework] == :minitest
+      use_rspec    if options[:framework] == :rspec
+
+      spinach = File.expand_path("bin/spinach")
+
+      run "#{ruby} #{spinach} #{options[:append]}"
+    end
+
     def ruby
       return @ruby if defined?(@ruby)
 

data/lib/spinach.rb

@@ -9,6 +9,7 @@ require_relative 'spinach/parser'
 require_relative 'spinach/dsl'
 require_relative 'spinach/feature_steps'
 require_relative 'spinach/reporter'
+require_relative 'spinach/orderers'
 require_relative 'spinach/cli'
 require_relative 'spinach/generators'
 require_relative 'spinach/auditor'

data/lib/spinach/cli.rb

@@ -131,6 +131,16 @@ module Spinach
             config[:reporter_classes] = names
           end
 
+          opts.on('--rand', "Randomize the order of features and scenarios") do
+            config[:orderer_class] = orderer_class(:random)
+          end
+
+          opts.on('--seed SEED', Integer,
+                  "Provide a seed for randomizing the order of features and scenarios") do |seed|
+            config[:orderer_class] = orderer_class(:random)
+            config[:seed]          = seed
+          end
+
           opts.on_tail('--fail-fast',
                        'Terminate the suite run on the first failure') do |class_name|
             config[:fail_fast] = true
@@ -164,7 +174,7 @@ and obsolete steps") do
     # Builds the class name to use an output reporter.
     #
     # @param [String] klass
-    #   The class name fo the reporter.
+    #   The class name of the reporter.
     #
     # @return [String]
     #   The full name of the reporter class.
@@ -177,5 +187,22 @@ and obsolete steps") do
     def reporter_class(klass)
       "Spinach::Reporter::" + Spinach::Support.camelize(klass)
     end
+
+    # Builds the class to use an orderer.
+    #
+    # @param [String] klass
+    #   The class name of the orderer.
+    #
+    # @return [String]
+    #   The full name of the orderer class.
+    #
+    # @example
+    #   orderer_class('random')
+    #   # => Spinach::Orderers::Random
+    #
+    # @api private
+    def orderer_class(klass)
+      "Spinach::Orderers::" + Spinach::Support.camelize(klass)
+    end
   end
 end

data/lib/spinach/config.rb

@@ -33,6 +33,8 @@ module Spinach
                 :save_and_open_page_on_failure,
                 :reporter_classes,
                 :reporter_options,
+                :orderer_class,
+                :seed,
                 :fail_fast,
                 :audit
 
@@ -66,6 +68,29 @@ module Spinach
       @reporter_options || {}
     end
 
+    # The "orderer class" holds the orderer class name
+    # Defaults to Spinach::Orderers::Default
+    #
+    # @return [orderer object]
+    #    The orderer that responds to specific messages.
+    #
+    # @api public
+    def orderer_class
+      @orderer_class || "Spinach::Orderers::Default"
+    end
+
+    # A randomization seed. This is what spinach uses for test run
+    # randomization, so if you call `Kernel.srand(Spinach.config.seed)`
+    # in your support environment file, not only will the test run
+    # order be guaranteed to be stable under a specific seed, all
+    # the Ruby-generated random numbers produced during your test
+    # run will also be stable under that seed.
+    #
+    # @api public
+    def seed
+      @seed ||= rand(0xFFFF)
+    end
+
     # The "step definitions path" holds the place where your feature step
     # classes will be searched for. Defaults to '#{features_path}/steps'
     #

data/lib/spinach/feature.rb

@@ -24,6 +24,14 @@ module Spinach
       lines_to_run.empty?
     end
 
+    # Identifier used by orderers.
+    #
+    # Needs to involve the relative file path so that the ordering
+    # a seed generates is stable across both runs and machines.
+    #
+    # @api public
+    alias ordering_id filename
+
     # Run the provided code for every step
     def each_step
       scenarios.each { |scenario| scenario.steps.each { |step| yield step } }

data/lib/spinach/orderers.rb

@@ -0,0 +1,2 @@
+require_relative 'orderers/default'
+require_relative 'orderers/random'

data/lib/spinach/orderers/default.rb

@@ -0,0 +1,25 @@
+module Spinach
+  module Orderers
+    class Default
+      def initialize(**options); end
+
+      # Appends any necessary report output (by default does nothing).
+      #
+      # @param [IO] io
+      #   Output buffer for report.
+      #
+      # @api public
+      def attach_summary(io); end
+
+      # Returns a reordered version of the provided array
+      #
+      # @param [Array] items
+      #   Items to order
+      #
+      # @api public
+      def order(items)
+        items
+      end
+    end
+  end
+end

data/lib/spinach/orderers/random.rb

@@ -0,0 +1,35 @@
+require 'digest'
+
+module Spinach
+  module Orderers
+    class Random
+      attr_reader :seed
+
+      def initialize(seed:)
+        @seed = seed.to_s
+      end
+
+      # Output the randomization seed in the report summary.
+      #
+      # @param [IO] io
+      #   Output buffer for report.
+      #
+      # @api public
+      def attach_summary(io)
+        io.puts("Randomized with seed #{seed}\n\n")
+      end
+
+      # Returns a reordered version of the provided array
+      #
+      # @param [Array] items
+      #   Items to order
+      #
+      # @api public
+      def order(items)
+        items.sort_by do |item|
+          Digest::MD5.hexdigest(seed + item.ordering_id)
+        end
+      end
+    end
+  end
+end

data/lib/spinach/reporter.rb

@@ -10,6 +10,7 @@ module Spinach
     def initialize(options = {})
       @errors = []
       @options = options
+      @orderer = options[:orderer]
       @undefined_features = []
       @successful_steps = []
       @undefined_steps = []

data/lib/spinach/reporter/progress.rb

@@ -8,7 +8,7 @@ module Spinach
     class Progress < Reporter
       include Reporting
 
-      # Initialitzes the runner
+      # Initializes the reporter
       #
       # @param [Hash] options
       #  Sets a custom output buffer by setting options[:output]

data/lib/spinach/reporter/reporting.rb

@@ -221,7 +221,9 @@ module Spinach
         error_summary      = format_summary(:red,    error_steps,      'Error')
 
         out.puts "Steps Summary: #{successful_summary}, #{pending_summary}, #{undefined_summary}, #{failed_summary}, #{error_summary}\n\n"
-        out.puts "Finished in #{Time.now - @start_time} seconds" if @start_time
+        out.puts "Finished in #{Time.now - @start_time} seconds\n\n" if @start_time
+
+        @orderer.attach_summary(out) if @orderer
       end
     end
   end

data/lib/spinach/reporter/stdout.rb

@@ -8,7 +8,7 @@ module Spinach
     class Stdout < Reporter
       include Reporting
 
-      # Initialitzes the runner
+      # Initializes the reporter
       #
       # @param [Hash] options
       #  Sets a custom output buffer by setting options[:output]

data/lib/spinach/runner.rb

@@ -30,10 +30,10 @@ module Spinach
     def initialize(filenames, options = {})
       @filenames = filenames
 
-      @step_definitions_path = options.delete(:step_definitions_path ) ||
+      @step_definitions_path = options.delete(:step_definitions_path) ||
         Spinach.config.step_definitions_path
 
-      @support_path = options.delete(:support_path ) ||
+      @support_path = options.delete(:support_path) ||
         Spinach.config.support_path
     end
 
@@ -42,7 +42,9 @@ module Spinach
     # @api public
     def init_reporters
       Spinach.config[:reporter_classes].each do |reporter_class|
-        reporter = Support.constantize(reporter_class).new(Spinach.config.reporter_options)
+        reporter_options = default_reporter_options.merge(Spinach.config.reporter_options)
+        reporter         = Support.constantize(reporter_class).new(reporter_options)
+
         reporter.bind
       end
     end
@@ -58,24 +60,12 @@ module Spinach
       require_frameworks
       init_reporters
 
-      features = filenames.map do |filename|
-        file, *lines = filename.split(":") # little more complex than just a "filename"
-
-        # FIXME Feature should be instantiated directly, not through an unrelated class method
-        feature          = Parser.open_file(file).parse
-        feature.filename = file
-
-        feature.lines_to_run = lines if lines.any?
-
-        feature
-      end
-
       suite_passed = true
 
       Spinach.hooks.run_before_run
 
-      features.each do |feature|
-        feature_passed = FeatureRunner.new(feature).run
+      features_to_run.each do |feature|
+        feature_passed = FeatureRunner.new(feature, orderer: orderer).run
         suite_passed &&= feature_passed
 
         break if fail_fast? && !feature_passed
@@ -139,11 +129,42 @@ module Spinach
       support_files + step_definition_files
     end
 
+    # The orderer for this run.
+    #
+    # @api public
+    def orderer
+      @orderer ||= Support.constantize(Spinach.config[:orderer_class]).new(
+        seed: Spinach.config.seed
+      )
+    end
+
+    # Default initialization options for the reporter
+    #
+    def default_reporter_options
+      {orderer: orderer}
+    end
+
     private
 
     def fail_fast?
       Spinach.config.fail_fast
     end
+
+    def features_to_run
+      unordered_features = filenames.map do |filename|
+        file, *lines = filename.split(":") # little more complex than just a "filename"
+
+        # FIXME Feature should be instantiated directly, not through an unrelated class method
+        feature          = Parser.open_file(file).parse
+        feature.filename = file
+
+        feature.lines_to_run = lines if lines.any?
+
+        feature
+      end
+
+      orderer.order(unordered_features)
+    end
   end
 end
 

data/lib/spinach/runner/feature_runner.rb

@@ -1,18 +1,20 @@
 require_relative '../tags_matcher'
+require 'spinach/orderers/default'
 
 module Spinach
   class Runner
     # A feature runner handles a particular feature run.
     #
     class FeatureRunner
-      attr_reader :feature
+      attr_reader :feature, :orderer
 
       # @param [GherkinRuby::AST::Feature] feature
       #   The feature to run.
       #
       # @api public
-      def initialize(feature)
+      def initialize(feature, orderer: Spinach::Orderers::Default.new)
         @feature = feature
+        @orderer = orderer
       end
 
       # @return [String]
@@ -80,7 +82,7 @@ module Spinach
       end
 
       def scenarios_to_run
-        feature.scenarios.select do |scenario|
+        unordered_scenarios = feature.scenarios.select do |scenario|
           has_a_tag_that_will_be_run = TagsMatcher.match(feature_tags + scenario.tags)
           on_a_line_that_will_be_run = if feature.run_every_scenario?
                                          true
@@ -90,6 +92,8 @@ module Spinach
 
           has_a_tag_that_will_be_run && on_a_line_that_will_be_run
         end
+
+        orderer.order(unordered_scenarios)
       end
     end
   end

data/lib/spinach/scenario.rb

@@ -9,5 +9,15 @@ module Spinach
       @tags    = []
       @lines   = []
     end
+
+    # Identifier used by orderers.
+    #
+    # Needs to involve the relative file path and line number so that the
+    # ordering a seed generates is stable across both runs and machines.
+    #
+    # @api public
+    def ordering_id
+      "#{feature.ordering_id}:#{lines.first}"
+    end
   end
 end

data/lib/spinach/version.rb

@@ -1,4 +1,4 @@
 module Spinach
   # Spinach version.
-  VERSION = "0.10.1"
+  VERSION = "0.11.0"
 end

data/spinach.gemspec

@@ -14,7 +14,7 @@ Gem::Specification.new do |gem|
   gem.add_runtime_dependency 'colorize'
   gem.add_runtime_dependency 'json'
   gem.add_development_dependency 'rake'
-  gem.add_development_dependency 'mocha', '~> 1.0'
+  gem.add_development_dependency 'mocha', "~> 1.5.0"
   gem.add_development_dependency 'sinatra'
   gem.add_development_dependency 'capybara'
   gem.add_development_dependency 'pry'
@@ -23,6 +23,8 @@ Gem::Specification.new do |gem|
   gem.add_development_dependency 'minitest', '< 5.0'
   gem.add_development_dependency 'fakefs', ">= 0.5.2"
 
+  gem.required_ruby_version = Gem::Requirement.new(">= 2.4".freeze)
+
   gem.executables   = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
   gem.files         = `git ls-files`.split("\n")
   gem.test_files    = `git ls-files -- {test,spec,features}/*`.split("\n")

data/test/spinach/cli_test.rb

@@ -89,6 +89,40 @@ tags:
       end
     end
 
+    describe 'rand' do
+      let(:config) { Spinach::Config.new }
+
+      before do
+        Spinach.stubs(:config).returns(config)
+        Spinach::Cli.new(%w(--rand)).options
+      end
+
+      it 'uses the Random orderer' do
+        config.orderer_class.must_equal 'Spinach::Orderers::Random'
+      end
+
+      it 'sets a numeric seed' do
+        config.seed.must_equal config.seed.to_i
+      end
+    end
+
+    describe 'seed' do
+      let(:config) { Spinach::Config.new }
+
+      before do
+        Spinach.stubs(:config).returns(config)
+        Spinach::Cli.new(%w(--seed 42)).options
+      end
+
+      it 'uses the random orderer' do
+        config.orderer_class.must_equal 'Spinach::Orderers::Random'
+      end
+
+      it 'sets the seed' do
+        config.seed.must_equal 42
+      end
+    end
+
     describe 'features_path' do
       %w{-f --features_path}.each do |opt|
         it 'sets the given features_path' do

data/test/spinach/config_test.rb

@@ -27,6 +27,28 @@ describe Spinach::Config do
    end
  end
 
+ describe '#orderer_class' do
+   it 'returns a default' do
+     subject[:orderer_class].must_equal "Spinach::Orderers::Default"
+   end
+
+   it 'can be overwritten' do
+     subject[:orderer_class] = "MyOwnOrderer"
+     subject[:orderer_class].must_equal "MyOwnOrderer"
+   end
+ end
+
+ describe '#seed' do
+   it 'has a default' do
+     subject[:seed].must_be_kind_of Integer
+   end
+
+   it 'can be overwritten' do
+     subject[:seed] = 54321
+     subject[:seed].must_equal 54321
+   end
+ end
+
  describe '#step_definitions_path' do
    it 'returns a default' do
      subject[:step_definitions_path].must_be_kind_of String

data/test/spinach/feature_test.rb

@@ -29,5 +29,15 @@ module Spinach
         end
       end
     end
+
+    describe "#ordering_id" do
+      subject { Feature.new }
+
+      before { subject.filename = "features/foo/bar.feature" }
+
+      it 'is the filename' do
+        subject.ordering_id.must_equal "features/foo/bar.feature"
+      end
+    end
   end
 end

data/test/spinach/hooks_test.rb

@@ -108,7 +108,7 @@ describe Spinach::Hooks do
         subject.on_tag('screenshot') do
           assertion = true
         end
-        subject.run_before_scenario(scenario)
+        subject.run_before_scenario(scenario, step_definitions)
         assertion.wont_equal true
       end
     end

data/test/spinach/orderers/default_test.rb

@@ -0,0 +1,31 @@
+require_relative '../../test_helper'
+
+describe Spinach::Orderers::Default do
+  let(:orderer) { Spinach::Orderers::Default.new }
+
+  describe "#attach_summary" do
+    let(:io) { StringIO.new }
+
+    it 'appends nothing' do
+      contents_before_running = io.string.dup
+
+      orderer.attach_summary(io)
+
+      io.string.must_equal contents_before_running
+    end
+  end
+
+  describe "#order" do
+    let(:items) { Array(1..10) }
+
+    it "doesn't change the order of the items" do
+      orderer.order(items).must_equal items
+    end
+  end
+
+  describe "#initialize" do
+    it "can be provided options without raising an error" do
+      Spinach::Orderers::Default.new(seed: "seed")
+    end
+  end
+end

data/test/spinach/orderers/random_test.rb

@@ -0,0 +1,39 @@
+require_relative '../../test_helper'
+
+describe Spinach::Orderers::Random do
+  let(:orderer) { Spinach::Orderers::Random.new(seed: Spinach.config.seed) }
+
+  describe "#attach_summary" do
+    let(:io) { StringIO.new }
+
+    it 'appends the seed' do
+      orderer.attach_summary(io)
+
+      io.string.must_match /Randomized\ with\ seed\ #{orderer.seed}/
+    end
+  end
+
+  describe "#order" do
+    Identifiable = Struct.new(:ordering_id)
+
+    let(:items) { (1..10).map { |n| Identifiable.new(n.to_s) } }
+
+    it "randomizes the items" do
+      orderer.order(items).wont_equal items
+    end
+
+    it "always randomizes items the same way with the same seed" do
+      orderer.order(items).must_equal orderer.order(items)
+    end
+  end
+
+  describe "#initialize" do
+    it "requires a seed parameter" do
+      proc {
+        Spinach::Orderers::Random.new
+      }.must_raise ArgumentError
+
+      Spinach::Orderers::Random.new(seed: 4)
+    end
+  end
+end

data/test/spinach/runner_test.rb

@@ -65,7 +65,10 @@ describe Spinach::Runner do
         config.reporter_options = {backtrace: true}
         reporter = stub
         reporter.stubs(:bind)
-        Spinach::Reporter::Stdout.expects(new: reporter).with(backtrace: true)
+        Spinach::Reporter::Stdout.expects(new: reporter).with(
+          backtrace: true,
+          **runner.default_reporter_options
+        )
         runner.init_reporters
       end
     end

data/test/spinach/scenario_test.rb

@@ -2,5 +2,19 @@ require 'test_helper'
 
 module Spinach
   describe Scenario do
+    describe "#ordering_id" do
+      let(:feature) { Feature.new }
+
+      subject { Scenario.new(feature) }
+
+      before do
+        feature.filename = "features/foo/bar.feature"
+        subject.lines    = Array(4..12)
+      end
+
+      it 'is the filename and starting line number' do
+        subject.ordering_id.must_equal "features/foo/bar.feature:4"
+      end
+    end
   end
 end

data/test/test_helper.rb

@@ -13,7 +13,7 @@ end
 
 require 'minitest/autorun'
 require 'minitest/spec'
-require 'mocha/mini_test'
+require 'mocha/minitest'
 require 'ostruct'
 require 'stringio'
 require 'pry'

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: spinach
 version: !ruby/object:Gem::Version
-  version: 0.10.1
+  version: 0.11.0
 platform: ruby
 authors:
 - Josep Jaume Rey
@@ -11,7 +11,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2018-02-21 00:00:00.000000000 Z
+date: 2020-04-20 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: gherkin-ruby
@@ -75,14 +75,14 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.0'
+        version: 1.5.0
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.0'
+        version: 1.5.0
 - !ruby/object:Gem::Dependency
   name: sinatra
   requirement: !ruby/object:Gem::Requirement
@@ -194,10 +194,10 @@ extensions: []
 extra_rdoc_files: []
 files:
 - ".document"
+- ".github/workflows/ci.yml"
 - ".gitignore"
 - ".ruby-gemset"
 - ".ruby-version"
-- ".travis.yml"
 - CHANGELOG.md
 - Gemfile
 - Guardfile
@@ -213,6 +213,7 @@ files:
 - features/fail_fast.feature
 - features/feature_name_guessing.feature
 - features/pending_steps.feature
+- features/randomization.feature
 - features/reporting/customized_reporter.feature
 - features/reporting/display_run_summary.feature
 - features/reporting/error_reporting.feature
@@ -230,6 +231,7 @@ files:
 - features/steps/fail_fast_option.rb
 - features/steps/feature_name_guessing.rb
 - features/steps/pending_steps.rb
+- features/steps/randomizing_features_scenarios.rb
 - features/steps/reporting/display_run_summary.rb
 - features/steps/reporting/error_reporting.rb
 - features/steps/reporting/pending_feature_reporting.rb
@@ -263,6 +265,9 @@ files:
 - lib/spinach/generators/step_generator.rb
 - lib/spinach/hookable.rb
 - lib/spinach/hooks.rb
+- lib/spinach/orderers.rb
+- lib/spinach/orderers/default.rb
+- lib/spinach/orderers/random.rb
 - lib/spinach/parser.rb
 - lib/spinach/parser/visitor.rb
 - lib/spinach/reporter.rb
@@ -293,6 +298,8 @@ files:
 - test/spinach/generators_test.rb
 - test/spinach/hookable_test.rb
 - test/spinach/hooks_test.rb
+- test/spinach/orderers/default_test.rb
+- test/spinach/orderers/random_test.rb
 - test/spinach/parser/visitor_test.rb
 - test/spinach/parser_test.rb
 - test/spinach/reporter/failure_file_test.rb
@@ -322,15 +329,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
-      version: '0'
+      version: '2.4'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubyforge_project: 
-rubygems_version: 2.5.1
+rubygems_version: 3.0.3
 signing_key: 
 specification_version: 4
 summary: Spinach is a BDD framework on top of gherkin
@@ -343,6 +349,7 @@ test_files:
 - features/fail_fast.feature
 - features/feature_name_guessing.feature
 - features/pending_steps.feature
+- features/randomization.feature
 - features/reporting/customized_reporter.feature
 - features/reporting/display_run_summary.feature
 - features/reporting/error_reporting.feature
@@ -360,6 +367,7 @@ test_files:
 - features/steps/fail_fast_option.rb
 - features/steps/feature_name_guessing.rb
 - features/steps/pending_steps.rb
+- features/steps/randomizing_features_scenarios.rb
 - features/steps/reporting/display_run_summary.rb
 - features/steps/reporting/error_reporting.rb
 - features/steps/reporting/pending_feature_reporting.rb
@@ -387,6 +395,8 @@ test_files:
 - test/spinach/generators_test.rb
 - test/spinach/hookable_test.rb
 - test/spinach/hooks_test.rb
+- test/spinach/orderers/default_test.rb
+- test/spinach/orderers/random_test.rb
 - test/spinach/parser/visitor_test.rb
 - test/spinach/parser_test.rb
 - test/spinach/reporter/failure_file_test.rb

data/.travis.yml

@@ -1,8 +0,0 @@
-rvm:
-  - 2.1.9
-  - 2.2.4
-  - 2.3.1
-  - jruby-9.1.2.0
-
-cache: bundler
-script: "bundle exec rake"