spinach 0.10.1 → 0.11.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
- SHA1:
3
- metadata.gz: 1f75603856e6e68fb9f7eb16a287062659034e02
4
- data.tar.gz: a6ea70aaa3ef522f10e6b5c729671c94c84bc67c
2
+ SHA256:
3
+ metadata.gz: 66296e07e2af7b9daa268623eb0e59ac1e90ee16855a3baf9eb63801be1338d2
4
+ data.tar.gz: b2239f90a08a88a7b9c0ca93d16103b816249122c60176ea094405f8d1f8ebf8
5
5
  SHA512:
6
- metadata.gz: c5f580a7a0bc2c9f4ce8901b78f1f63da2a72d14f8233ade3ebd423f6bf0989441b318970266610422b6e53de854fd6cf1ce868f53e80064f6606396d9c39c3f
7
- data.tar.gz: 349c9aa0672871576ea8ff8f86944c24816048ccbee2ca3a360b446feed8d6ede0ab2671a8db2505ea4cfd80a0f3e5e66f55578e88d69db215981d37970c06a3
6
+ metadata.gz: 909a2ec0656f8991d9ab05a3a26864544da914475de5d8205ada4a20b3041c8cddb81d33319329252a440ec53e32852ff2daf0a0364fe9d87b2c1bff815725c4
7
+ data.tar.gz: b08372a0eb16eb49f0ca4972f8a412307ea9cd0eba8eab48c848ff849700bad2dc79444f6eb73d3830ed67941c0ce3efa44803658aa71da453a10fa00fa114c1
@@ -0,0 +1,31 @@
1
+ name: "Tests"
2
+
3
+ on:
4
+ push:
5
+ branches:
6
+ - master
7
+ pull_request:
8
+ branches:
9
+ - "*"
10
+
11
+ env:
12
+ CI: "true"
13
+
14
+ jobs:
15
+ main:
16
+ name: Tests
17
+ runs-on: ubuntu-latest
18
+ strategy:
19
+ matrix:
20
+ ruby: [2.4, 2.5, 2.6, 2.7, jruby]
21
+ fail-fast: false
22
+ steps:
23
+ - uses: actions/checkout@v2.0.0
24
+ with:
25
+ fetch-depth: 1
26
+ - uses: ruby/setup-ruby@master
27
+ with:
28
+ ruby-version: ${{ matrix.ruby }}
29
+ - run: bundle install --jobs 4 --retry 3
30
+ name: Install Ruby deps
31
+ - run: bundle exec rake
@@ -1 +1 @@
1
- 2.3.1
1
+ 2.7.1
@@ -1,8 +1,9 @@
1
1
  # Spinach - BDD framework on top of Gherkin
2
- [![Gem Version](https://badge.fury.io/rb/spinach.png)](http://badge.fury.io/rb/spinach)
3
- [![Build Status](https://secure.travis-ci.org/codegram/spinach.png)](http://travis-ci.org/codegram/spinach)
4
- [![Dependency Status](https://gemnasium.com/codegram/spinach.png)](http://gemnasium.com/codegram/spinach)
2
+
3
+ [![Gem Version](https://badge.fury.io/rb/spinach.svg)](https://badge.fury.io/rb/spinach)
4
+ ![Tests](https://github.com/codegram/spinach/workflows/Tests/badge.svg)
5
5
  [![Coverage Status](https://coveralls.io/repos/codegram/spinach/badge.png?branch=master)](https://coveralls.io/r/codegram/spinach)
6
+ ![](https://ruby-gem-downloads-badge.herokuapp.com/rails)
6
7
 
7
8
  Spinach is a high-level BDD framework that leverages the expressive
8
9
  [Gherkin language][gherkin] (used by [Cucumber][cucumber]) to help you define
@@ -10,13 +11,13 @@ executable specifications of your application or library's acceptance criteria.
10
11
 
11
12
  Conceived as an alternative to Cucumber, here are some of its design goals:
12
13
 
13
- * Step maintainability: since features map to their own classes, their steps are
14
+ - Step maintainability: since features map to their own classes, their steps are
14
15
  just methods of that class. This encourages step encapsulation.
15
16
 
16
- * Step reusability: In case you want to reuse steps across features, you can
17
+ - Step reusability: In case you want to reuse steps across features, you can
17
18
  always wrap those in plain ol' Ruby modules.
18
19
 
19
- Spinach is tested against **2.0, 2.1, 2.2 and 2.3** as well as **JRuby 9000**.
20
+ Spinach is tested against Ruby MRI **2.4, 2.5, 2.6 and 2.7** as well as **latest JRuby 9000**.
20
21
 
21
22
  ## Getting started
22
23
 
@@ -184,7 +185,7 @@ end
184
185
  ## Audit
185
186
 
186
187
  Over time, the definitions of your features will change. When you add, remove
187
- or change steps in the feature files, you can easily audit your existing step
188
+ or change steps in the feature files, you can easily audit your existing step
188
189
  files with:
189
190
 
190
191
  ```shell
@@ -198,12 +199,11 @@ This does not modify the step files, so you will need to paste the boilerplate
198
199
  into the appropriate places. If a new feature file is detected, you will be
199
200
  asked to run `spinach --generate` beforehand.
200
201
 
201
- **Important**: If auditing individual files, common steps (as above) may be
202
- reported as unused when they are actually used in a feature file that is not
202
+ **Important**: If auditing individual files, common steps (as above) may be
203
+ reported as unused when they are actually used in a feature file that is not
203
204
  currently being audited. To avoid this, run the audit with no arguments to
204
205
  audit all step files simultaneously.
205
206
 
206
-
207
207
  ## Tags
208
208
 
209
209
  Feature and Scenarios can be marked with tags in the form: `@tag`. Tags can be
@@ -238,19 +238,19 @@ Feature: So something great
238
238
  Scenario: Ensure no regression on this
239
239
  ```
240
240
 
241
- Then you can run all Scenarios in your suite related to `@feat-1` using:
241
+ Then you can run all Scenarios in your suite related to `@feat-1` using:
242
242
 
243
243
  ```shell
244
244
  $ spinach --tags @feat-1
245
245
  ```
246
246
 
247
- Or only Scenarios related to `@feat-1` and `@bug-12` using:
247
+ Or only Scenarios related to `@feat-1` and `@bug-12` using:
248
248
 
249
249
  ```shell
250
250
  $ spinach --tags @feat-1,@bug-12
251
251
  ```
252
252
 
253
- Or only Scenarios related to `@feat-1` excluding `@bug-12` using:
253
+ Or only Scenarios related to `@feat-1` excluding `@bug-12` using:
254
254
 
255
255
  ```shell
256
256
  $ spinach --tags @feat-1,~@bug-12
@@ -292,7 +292,7 @@ Full hook documentation is here:
292
292
 
293
293
  ## Local Before and After Hooks
294
294
 
295
- Sometimes it feels awkward to add steps into feature file just because you need to do some test setup and cleanup. And it is equally awkward to add a global hooks for this purpose. For example, if you want to add a session timeout feature, to do so, you want to set the session timeout time to 1 second just for this feature, and put the normal timeout back after this feature. It doesn't make sense to add two steps in the feature file just to change the session timeout value. In this scenario, a ```before``` and ```after``` blocks are perfect for this kind of tasks. Below is an example implementation:
295
+ Sometimes it feels awkward to add steps into feature file just because you need to do some test setup and cleanup. And it is equally awkward to add a global hooks for this purpose. For example, if you want to add a session timeout feature, to do so, you want to set the session timeout time to 1 second just for this feature, and put the normal timeout back after this feature. It doesn't make sense to add two steps in the feature file just to change the session timeout value. In this scenario, a `before` and `after` blocks are perfect for this kind of tasks. Below is an example implementation:
296
296
 
297
297
  ```ruby
298
298
  class Spinach::Features::SessionTimeout < Spinach::FeatureSteps
@@ -329,8 +329,8 @@ When no reporter is specified, `stdout` will be used by default.
329
329
 
330
330
  Other reporters:
331
331
 
332
- * For a console reporter with no colors, try [spinach-console-reporter][spinach-console-reporter] (to be used with Jenkins)
333
- * For a rerun reporter, try [spinach-rerun-reporter][spinach-rerun-reporter] (writes failed scenarios in a file)
332
+ - For a console reporter with no colors, try [spinach-console-reporter][spinach-console-reporter] (to be used with Jenkins)
333
+ - For a rerun reporter, try [spinach-rerun-reporter][spinach-rerun-reporter] (writes failed scenarios in a file)
334
334
 
335
335
  ## Wanna use it with Rails 3?
336
336
 
@@ -342,47 +342,46 @@ Check out our [spinach-sinatra demo](https://github.com/codegram/spinach-sinatra
342
342
 
343
343
  ## Resources
344
344
 
345
- * [Landing page](http://codegram.github.com/spinach)
346
- * [Slides](http://codegram.github.com/spinach-presentation)
347
- * [Blog post](http://blog.codegram.com/2011/10/how-to-achieve-more-clean-encapsulated-modular-step-definitions-with-spinach)
348
- * [API Documentation](http://rubydoc.info/github/codegram/spinach/master/frames)
349
- * [Google group](https://groups.google.com/forum/#!forum/spinach_bdd)
345
+ - [Landing page](http://codegram.github.com/spinach)
346
+ - [Slides](http://codegram.github.com/spinach-presentation)
347
+ - [Blog post](http://blog.codegram.com/2011/10/how-to-achieve-more-clean-encapsulated-modular-step-definitions-with-spinach)
348
+ - [API Documentation](http://rubydoc.info/github/codegram/spinach/master/frames)
349
+ - [Google group](https://groups.google.com/forum/#!forum/spinach_bdd)
350
350
 
351
351
  ### Related gems
352
352
 
353
- * [guard-spinach](http://github.com/codegram/guard-spinach)
354
- * [spinach-rails](http://github.com/codegram/spinach-rails)
355
- * [spinach-console-reporter][spinach-console-reporter] (to be used with Jenkins)
356
- * [spinach-rerun-reporter][spinach-rerun-reporter] (writes failed scenarios in a file)
357
- * [spring-commands-spinach](https://github.com/jvanbaarsen/spring-commands-spinach) (to be used with spring)
353
+ - [guard-spinach](http://github.com/codegram/guard-spinach)
354
+ - [spinach-rails](http://github.com/codegram/spinach-rails)
355
+ - [spinach-console-reporter][spinach-console-reporter] (to be used with Jenkins)
356
+ - [spinach-rerun-reporter][spinach-rerun-reporter] (writes failed scenarios in a file)
357
+ - [spring-commands-spinach](https://github.com/jvanbaarsen/spring-commands-spinach) (to be used with spring)
358
358
 
359
359
  ### Demos
360
360
 
361
- * [spinach rails demo](https://github.com/codegram/spinach-rails-demo)
362
- * [spinach sinatra demo](https://github.com/codegram/spinach-sinatra-demo)
363
- * [simple todo Rails app](https://github.com/codegram/tasca-spinach-demo)
361
+ - [spinach rails demo](https://github.com/codegram/spinach-rails-demo)
362
+ - [spinach sinatra demo](https://github.com/codegram/spinach-sinatra-demo)
363
+ - [simple todo Rails app](https://github.com/codegram/tasca-spinach-demo)
364
364
 
365
365
  ## Contributing
366
366
 
367
- * [List of spinach contributors](https://github.com/codegram/spinach/contributors)
367
+ - [List of spinach contributors](https://github.com/codegram/spinach/contributors)
368
368
 
369
369
  You can easily contribute to Spinach. Its codebase is simple and
370
370
  [extensively documented][documentation].
371
371
 
372
- * Fork the project.
373
- * Make your feature addition or bug fix.
374
- * Add specs for it. This is important so we don't break it in a future
372
+ - Fork the project.
373
+ - Make your feature addition or bug fix.
374
+ - Add specs for it. This is important so we don't break it in a future
375
375
  version unintentionally.
376
- * Commit, do not mess with rakefile, version, or history.
376
+ - Commit, do not mess with rakefile, version, or history.
377
377
  If you want to have your own version, that is fine but bump version
378
378
  in a commit by itself I can ignore when I pull.
379
- * Send me a pull request. Bonus points for topic branches.
379
+ - Send me a pull request. Bonus points for topic branches.
380
380
 
381
381
  ## License
382
382
 
383
383
  MIT (Expat) License. Copyright 2011-2016 [Codegram Technologies](http://codegram.com)
384
384
 
385
-
386
385
  [gherkin]: http://github.com/codegram/gherkin-ruby
387
386
  [cucumber]: http://github.com/cucumber/cucumber
388
387
  [documentation]: http://rubydoc.info/github/codegram/spinach/master/frames
data/Rakefile CHANGED
@@ -11,7 +11,7 @@ end
11
11
 
12
12
  desc 'Run spinach features'
13
13
  task :spinach do
14
- exec "bin/spinach"
14
+ exec "bin/spinach --rand"
15
15
  end
16
16
 
17
17
  task :default => [:test, :spinach]
@@ -0,0 +1,16 @@
1
+ Feature: Randomizing Features & Scenarios
2
+ In order to ensure my tests aren't dependent
3
+ As a developer
4
+ I want spinach to randomize features and scenarios (but not steps)
5
+
6
+ Scenario: Randomizing the run without specifying a seed
7
+ Given I have 2 features with 2 scenarios each
8
+ When I randomize the run without specifying a seed
9
+ Then The features and scenarios are run
10
+ And The runner output shows a seed
11
+
12
+ Scenario: Specifying the seed
13
+ Given I have 2 features with 2 scenarios each
14
+ When I specify the seed for the run
15
+ Then The features and scenarios are run in a different order
16
+ And The runner output shows the seed
@@ -1,9 +1,25 @@
1
1
  Feature: Display run summary
2
2
  As a developer
3
3
  I want spinach to display a summary of steps statuses
4
- So I can easyly know general features status
4
+ So I can easily know general features status
5
5
 
6
- Scenario: Display run summary at the end of features run
6
+ Scenario: Display run summary at the end of features run without randomization
7
7
  Given I have a feature that has some successful, undefined, failed and error steps
8
- When I run it
8
+
9
+ When I run it without randomization
10
+ Then I should see a summary with steps status information
11
+ And I shouldn't see a randomization seed
12
+
13
+ Scenario: Display run summary at the end of features run with randomization
14
+ Given I have a feature that has some successful, undefined, failed and error steps
15
+
16
+ When I run it with randomization
17
+ Then I should see a summary with steps status information
18
+ And I should see a randomization seed
19
+
20
+ Scenario: Display run summary at the end of features run with a randomization seed
21
+ Given I have a feature that has some successful, undefined, failed and error steps
22
+
23
+ When I run it with a specific randomization seed
9
24
  Then I should see a summary with steps status information
25
+ And I should see that specific randomization seed
@@ -0,0 +1,68 @@
1
+ class Spinach::Features::RandomizingFeaturesScenarios < Spinach::FeatureSteps
2
+ include Integration::SpinachRunner
3
+
4
+ step 'I have 2 features with 2 scenarios each' do
5
+ write_file 'features/success_a.feature', <<-FEATURE
6
+ Feature: Success A
7
+ Scenario: A1
8
+ Then a1
9
+ Scenario: A2
10
+ Then a2
11
+ FEATURE
12
+
13
+ write_file 'features/steps/success_a.rb', <<-STEPS
14
+ class Spinach::Features::SuccessA < Spinach::FeatureSteps
15
+ step 'a1' do; end
16
+ step 'a2' do; end
17
+ end
18
+ STEPS
19
+
20
+ write_file 'features/success_b.feature', <<-FEATURE
21
+ Feature: Success B
22
+ Scenario: B1
23
+ Then b1
24
+ Scenario: B2
25
+ Then b2
26
+ FEATURE
27
+
28
+ write_file 'features/steps/success_b.rb', <<-STEPS
29
+ class Spinach::Features::SuccessB < Spinach::FeatureSteps
30
+ step 'b1' do; end
31
+ step 'b2' do; end
32
+ end
33
+ STEPS
34
+ end
35
+
36
+ step 'I randomize the run without specifying a seed' do
37
+ run_spinach({append: "--rand"})
38
+ end
39
+
40
+ step 'I specify the seed for the run' do
41
+ # Reverse order (A2 A1 B2 B1) is the only way I can show that
42
+ # scenarios and features are randomized by the seed when the
43
+ # example has 2 features each with 2 scenarios. I tried seeds
44
+ # until I found one that ordered the test in that order.
45
+ @seed = 1
46
+
47
+ run_spinach({append: "--seed #{@seed}"})
48
+ end
49
+
50
+ step 'The features and scenarios are run' do
51
+ @stdout.must_include("A1")
52
+ @stdout.must_include("A2")
53
+ @stdout.must_include("B1")
54
+ @stdout.must_include("B2")
55
+ end
56
+
57
+ step 'The features and scenarios are run in a different order' do
58
+ @stdout.must_match(/B2.*B1.*A2.*A1/m)
59
+ end
60
+
61
+ step 'The runner output shows a seed' do
62
+ @stdout.must_match(/^Randomized with seed \d*$/)
63
+ end
64
+
65
+ step 'The runner output shows the seed' do
66
+ @stdout.must_match(/^Randomized with seed #{@seed}$/)
67
+ end
68
+ end
@@ -62,7 +62,7 @@ Feature: A test feature
62
62
  @feature = "features/test_feature.feature"
63
63
  end
64
64
 
65
- When "I run it" do
65
+ When "I run it without randomization" do
66
66
  run_feature @feature
67
67
  end
68
68
 
@@ -71,4 +71,32 @@ Feature: A test feature
71
71
  /Summary:.*4.*Successful.*1.*Undefined.*1.*Failed.*1.*Error/
72
72
  )
73
73
  end
74
+
75
+ And "I shouldn't see a randomization seed" do
76
+ @stdout.wont_match(
77
+ /Randomized\ with\ seed\ \d+/
78
+ )
79
+ end
80
+
81
+ When "I run it with randomization" do
82
+ run_feature @feature, {append: "--rand"}
83
+ end
84
+
85
+ And "I should see a randomization seed" do
86
+ @stdout.must_match(
87
+ /Randomized\ with\ seed\ \d+/
88
+ )
89
+ end
90
+
91
+ When "I run it with a specific randomization seed" do
92
+ @seed = rand(0xFFFF)
93
+
94
+ run_feature @feature, {append: "--seed #{@seed}"}
95
+ end
96
+
97
+ And "I should see that specific randomization seed" do
98
+ @stdout.must_match(
99
+ /Randomized\ with\ seed\ #{@seed}/
100
+ )
101
+ end
74
102
  end
@@ -23,6 +23,17 @@ module Integration
23
23
  run "#{ruby} #{spinach} #{feature} #{options[:append]}", options[:env]
24
24
  end
25
25
 
26
+ def run_spinach(options = {})
27
+ options[:framework] ||= :minitest
28
+
29
+ use_minitest if options[:framework] == :minitest
30
+ use_rspec if options[:framework] == :rspec
31
+
32
+ spinach = File.expand_path("bin/spinach")
33
+
34
+ run "#{ruby} #{spinach} #{options[:append]}"
35
+ end
36
+
26
37
  def ruby
27
38
  return @ruby if defined?(@ruby)
28
39
 
@@ -9,6 +9,7 @@ require_relative 'spinach/parser'
9
9
  require_relative 'spinach/dsl'
10
10
  require_relative 'spinach/feature_steps'
11
11
  require_relative 'spinach/reporter'
12
+ require_relative 'spinach/orderers'
12
13
  require_relative 'spinach/cli'
13
14
  require_relative 'spinach/generators'
14
15
  require_relative 'spinach/auditor'
@@ -131,6 +131,16 @@ module Spinach
131
131
  config[:reporter_classes] = names
132
132
  end
133
133
 
134
+ opts.on('--rand', "Randomize the order of features and scenarios") do
135
+ config[:orderer_class] = orderer_class(:random)
136
+ end
137
+
138
+ opts.on('--seed SEED', Integer,
139
+ "Provide a seed for randomizing the order of features and scenarios") do |seed|
140
+ config[:orderer_class] = orderer_class(:random)
141
+ config[:seed] = seed
142
+ end
143
+
134
144
  opts.on_tail('--fail-fast',
135
145
  'Terminate the suite run on the first failure') do |class_name|
136
146
  config[:fail_fast] = true
@@ -164,7 +174,7 @@ and obsolete steps") do
164
174
  # Builds the class name to use an output reporter.
165
175
  #
166
176
  # @param [String] klass
167
- # The class name fo the reporter.
177
+ # The class name of the reporter.
168
178
  #
169
179
  # @return [String]
170
180
  # The full name of the reporter class.
@@ -177,5 +187,22 @@ and obsolete steps") do
177
187
  def reporter_class(klass)
178
188
  "Spinach::Reporter::" + Spinach::Support.camelize(klass)
179
189
  end
190
+
191
+ # Builds the class to use an orderer.
192
+ #
193
+ # @param [String] klass
194
+ # The class name of the orderer.
195
+ #
196
+ # @return [String]
197
+ # The full name of the orderer class.
198
+ #
199
+ # @example
200
+ # orderer_class('random')
201
+ # # => Spinach::Orderers::Random
202
+ #
203
+ # @api private
204
+ def orderer_class(klass)
205
+ "Spinach::Orderers::" + Spinach::Support.camelize(klass)
206
+ end
180
207
  end
181
208
  end
@@ -33,6 +33,8 @@ module Spinach
33
33
  :save_and_open_page_on_failure,
34
34
  :reporter_classes,
35
35
  :reporter_options,
36
+ :orderer_class,
37
+ :seed,
36
38
  :fail_fast,
37
39
  :audit
38
40
 
@@ -66,6 +68,29 @@ module Spinach
66
68
  @reporter_options || {}
67
69
  end
68
70
 
71
+ # The "orderer class" holds the orderer class name
72
+ # Defaults to Spinach::Orderers::Default
73
+ #
74
+ # @return [orderer object]
75
+ # The orderer that responds to specific messages.
76
+ #
77
+ # @api public
78
+ def orderer_class
79
+ @orderer_class || "Spinach::Orderers::Default"
80
+ end
81
+
82
+ # A randomization seed. This is what spinach uses for test run
83
+ # randomization, so if you call `Kernel.srand(Spinach.config.seed)`
84
+ # in your support environment file, not only will the test run
85
+ # order be guaranteed to be stable under a specific seed, all
86
+ # the Ruby-generated random numbers produced during your test
87
+ # run will also be stable under that seed.
88
+ #
89
+ # @api public
90
+ def seed
91
+ @seed ||= rand(0xFFFF)
92
+ end
93
+
69
94
  # The "step definitions path" holds the place where your feature step
70
95
  # classes will be searched for. Defaults to '#{features_path}/steps'
71
96
  #
@@ -24,6 +24,14 @@ module Spinach
24
24
  lines_to_run.empty?
25
25
  end
26
26
 
27
+ # Identifier used by orderers.
28
+ #
29
+ # Needs to involve the relative file path so that the ordering
30
+ # a seed generates is stable across both runs and machines.
31
+ #
32
+ # @api public
33
+ alias ordering_id filename
34
+
27
35
  # Run the provided code for every step
28
36
  def each_step
29
37
  scenarios.each { |scenario| scenario.steps.each { |step| yield step } }
@@ -0,0 +1,2 @@
1
+ require_relative 'orderers/default'
2
+ require_relative 'orderers/random'
@@ -0,0 +1,25 @@
1
+ module Spinach
2
+ module Orderers
3
+ class Default
4
+ def initialize(**options); end
5
+
6
+ # Appends any necessary report output (by default does nothing).
7
+ #
8
+ # @param [IO] io
9
+ # Output buffer for report.
10
+ #
11
+ # @api public
12
+ def attach_summary(io); end
13
+
14
+ # Returns a reordered version of the provided array
15
+ #
16
+ # @param [Array] items
17
+ # Items to order
18
+ #
19
+ # @api public
20
+ def order(items)
21
+ items
22
+ end
23
+ end
24
+ end
25
+ end
@@ -0,0 +1,35 @@
1
+ require 'digest'
2
+
3
+ module Spinach
4
+ module Orderers
5
+ class Random
6
+ attr_reader :seed
7
+
8
+ def initialize(seed:)
9
+ @seed = seed.to_s
10
+ end
11
+
12
+ # Output the randomization seed in the report summary.
13
+ #
14
+ # @param [IO] io
15
+ # Output buffer for report.
16
+ #
17
+ # @api public
18
+ def attach_summary(io)
19
+ io.puts("Randomized with seed #{seed}\n\n")
20
+ end
21
+
22
+ # Returns a reordered version of the provided array
23
+ #
24
+ # @param [Array] items
25
+ # Items to order
26
+ #
27
+ # @api public
28
+ def order(items)
29
+ items.sort_by do |item|
30
+ Digest::MD5.hexdigest(seed + item.ordering_id)
31
+ end
32
+ end
33
+ end
34
+ end
35
+ end
@@ -10,6 +10,7 @@ module Spinach
10
10
  def initialize(options = {})
11
11
  @errors = []
12
12
  @options = options
13
+ @orderer = options[:orderer]
13
14
  @undefined_features = []
14
15
  @successful_steps = []
15
16
  @undefined_steps = []
@@ -8,7 +8,7 @@ module Spinach
8
8
  class Progress < Reporter
9
9
  include Reporting
10
10
 
11
- # Initialitzes the runner
11
+ # Initializes the reporter
12
12
  #
13
13
  # @param [Hash] options
14
14
  # Sets a custom output buffer by setting options[:output]
@@ -221,7 +221,9 @@ module Spinach
221
221
  error_summary = format_summary(:red, error_steps, 'Error')
222
222
 
223
223
  out.puts "Steps Summary: #{successful_summary}, #{pending_summary}, #{undefined_summary}, #{failed_summary}, #{error_summary}\n\n"
224
- out.puts "Finished in #{Time.now - @start_time} seconds" if @start_time
224
+ out.puts "Finished in #{Time.now - @start_time} seconds\n\n" if @start_time
225
+
226
+ @orderer.attach_summary(out) if @orderer
225
227
  end
226
228
  end
227
229
  end
@@ -8,7 +8,7 @@ module Spinach
8
8
  class Stdout < Reporter
9
9
  include Reporting
10
10
 
11
- # Initialitzes the runner
11
+ # Initializes the reporter
12
12
  #
13
13
  # @param [Hash] options
14
14
  # Sets a custom output buffer by setting options[:output]
@@ -30,10 +30,10 @@ module Spinach
30
30
  def initialize(filenames, options = {})
31
31
  @filenames = filenames
32
32
 
33
- @step_definitions_path = options.delete(:step_definitions_path ) ||
33
+ @step_definitions_path = options.delete(:step_definitions_path) ||
34
34
  Spinach.config.step_definitions_path
35
35
 
36
- @support_path = options.delete(:support_path ) ||
36
+ @support_path = options.delete(:support_path) ||
37
37
  Spinach.config.support_path
38
38
  end
39
39
 
@@ -42,7 +42,9 @@ module Spinach
42
42
  # @api public
43
43
  def init_reporters
44
44
  Spinach.config[:reporter_classes].each do |reporter_class|
45
- reporter = Support.constantize(reporter_class).new(Spinach.config.reporter_options)
45
+ reporter_options = default_reporter_options.merge(Spinach.config.reporter_options)
46
+ reporter = Support.constantize(reporter_class).new(reporter_options)
47
+
46
48
  reporter.bind
47
49
  end
48
50
  end
@@ -58,24 +60,12 @@ module Spinach
58
60
  require_frameworks
59
61
  init_reporters
60
62
 
61
- features = filenames.map do |filename|
62
- file, *lines = filename.split(":") # little more complex than just a "filename"
63
-
64
- # FIXME Feature should be instantiated directly, not through an unrelated class method
65
- feature = Parser.open_file(file).parse
66
- feature.filename = file
67
-
68
- feature.lines_to_run = lines if lines.any?
69
-
70
- feature
71
- end
72
-
73
63
  suite_passed = true
74
64
 
75
65
  Spinach.hooks.run_before_run
76
66
 
77
- features.each do |feature|
78
- feature_passed = FeatureRunner.new(feature).run
67
+ features_to_run.each do |feature|
68
+ feature_passed = FeatureRunner.new(feature, orderer: orderer).run
79
69
  suite_passed &&= feature_passed
80
70
 
81
71
  break if fail_fast? && !feature_passed
@@ -139,11 +129,42 @@ module Spinach
139
129
  support_files + step_definition_files
140
130
  end
141
131
 
132
+ # The orderer for this run.
133
+ #
134
+ # @api public
135
+ def orderer
136
+ @orderer ||= Support.constantize(Spinach.config[:orderer_class]).new(
137
+ seed: Spinach.config.seed
138
+ )
139
+ end
140
+
141
+ # Default initialization options for the reporter
142
+ #
143
+ def default_reporter_options
144
+ {orderer: orderer}
145
+ end
146
+
142
147
  private
143
148
 
144
149
  def fail_fast?
145
150
  Spinach.config.fail_fast
146
151
  end
152
+
153
+ def features_to_run
154
+ unordered_features = filenames.map do |filename|
155
+ file, *lines = filename.split(":") # little more complex than just a "filename"
156
+
157
+ # FIXME Feature should be instantiated directly, not through an unrelated class method
158
+ feature = Parser.open_file(file).parse
159
+ feature.filename = file
160
+
161
+ feature.lines_to_run = lines if lines.any?
162
+
163
+ feature
164
+ end
165
+
166
+ orderer.order(unordered_features)
167
+ end
147
168
  end
148
169
  end
149
170
 
@@ -1,18 +1,20 @@
1
1
  require_relative '../tags_matcher'
2
+ require 'spinach/orderers/default'
2
3
 
3
4
  module Spinach
4
5
  class Runner
5
6
  # A feature runner handles a particular feature run.
6
7
  #
7
8
  class FeatureRunner
8
- attr_reader :feature
9
+ attr_reader :feature, :orderer
9
10
 
10
11
  # @param [GherkinRuby::AST::Feature] feature
11
12
  # The feature to run.
12
13
  #
13
14
  # @api public
14
- def initialize(feature)
15
+ def initialize(feature, orderer: Spinach::Orderers::Default.new)
15
16
  @feature = feature
17
+ @orderer = orderer
16
18
  end
17
19
 
18
20
  # @return [String]
@@ -80,7 +82,7 @@ module Spinach
80
82
  end
81
83
 
82
84
  def scenarios_to_run
83
- feature.scenarios.select do |scenario|
85
+ unordered_scenarios = feature.scenarios.select do |scenario|
84
86
  has_a_tag_that_will_be_run = TagsMatcher.match(feature_tags + scenario.tags)
85
87
  on_a_line_that_will_be_run = if feature.run_every_scenario?
86
88
  true
@@ -90,6 +92,8 @@ module Spinach
90
92
 
91
93
  has_a_tag_that_will_be_run && on_a_line_that_will_be_run
92
94
  end
95
+
96
+ orderer.order(unordered_scenarios)
93
97
  end
94
98
  end
95
99
  end
@@ -9,5 +9,15 @@ module Spinach
9
9
  @tags = []
10
10
  @lines = []
11
11
  end
12
+
13
+ # Identifier used by orderers.
14
+ #
15
+ # Needs to involve the relative file path and line number so that the
16
+ # ordering a seed generates is stable across both runs and machines.
17
+ #
18
+ # @api public
19
+ def ordering_id
20
+ "#{feature.ordering_id}:#{lines.first}"
21
+ end
12
22
  end
13
23
  end
@@ -1,4 +1,4 @@
1
1
  module Spinach
2
2
  # Spinach version.
3
- VERSION = "0.10.1"
3
+ VERSION = "0.11.0"
4
4
  end
@@ -14,7 +14,7 @@ Gem::Specification.new do |gem|
14
14
  gem.add_runtime_dependency 'colorize'
15
15
  gem.add_runtime_dependency 'json'
16
16
  gem.add_development_dependency 'rake'
17
- gem.add_development_dependency 'mocha', '~> 1.0'
17
+ gem.add_development_dependency 'mocha', "~> 1.5.0"
18
18
  gem.add_development_dependency 'sinatra'
19
19
  gem.add_development_dependency 'capybara'
20
20
  gem.add_development_dependency 'pry'
@@ -23,6 +23,8 @@ Gem::Specification.new do |gem|
23
23
  gem.add_development_dependency 'minitest', '< 5.0'
24
24
  gem.add_development_dependency 'fakefs', ">= 0.5.2"
25
25
 
26
+ gem.required_ruby_version = Gem::Requirement.new(">= 2.4".freeze)
27
+
26
28
  gem.executables = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
27
29
  gem.files = `git ls-files`.split("\n")
28
30
  gem.test_files = `git ls-files -- {test,spec,features}/*`.split("\n")
@@ -89,6 +89,40 @@ tags:
89
89
  end
90
90
  end
91
91
 
92
+ describe 'rand' do
93
+ let(:config) { Spinach::Config.new }
94
+
95
+ before do
96
+ Spinach.stubs(:config).returns(config)
97
+ Spinach::Cli.new(%w(--rand)).options
98
+ end
99
+
100
+ it 'uses the Random orderer' do
101
+ config.orderer_class.must_equal 'Spinach::Orderers::Random'
102
+ end
103
+
104
+ it 'sets a numeric seed' do
105
+ config.seed.must_equal config.seed.to_i
106
+ end
107
+ end
108
+
109
+ describe 'seed' do
110
+ let(:config) { Spinach::Config.new }
111
+
112
+ before do
113
+ Spinach.stubs(:config).returns(config)
114
+ Spinach::Cli.new(%w(--seed 42)).options
115
+ end
116
+
117
+ it 'uses the random orderer' do
118
+ config.orderer_class.must_equal 'Spinach::Orderers::Random'
119
+ end
120
+
121
+ it 'sets the seed' do
122
+ config.seed.must_equal 42
123
+ end
124
+ end
125
+
92
126
  describe 'features_path' do
93
127
  %w{-f --features_path}.each do |opt|
94
128
  it 'sets the given features_path' do
@@ -27,6 +27,28 @@ describe Spinach::Config do
27
27
  end
28
28
  end
29
29
 
30
+ describe '#orderer_class' do
31
+ it 'returns a default' do
32
+ subject[:orderer_class].must_equal "Spinach::Orderers::Default"
33
+ end
34
+
35
+ it 'can be overwritten' do
36
+ subject[:orderer_class] = "MyOwnOrderer"
37
+ subject[:orderer_class].must_equal "MyOwnOrderer"
38
+ end
39
+ end
40
+
41
+ describe '#seed' do
42
+ it 'has a default' do
43
+ subject[:seed].must_be_kind_of Integer
44
+ end
45
+
46
+ it 'can be overwritten' do
47
+ subject[:seed] = 54321
48
+ subject[:seed].must_equal 54321
49
+ end
50
+ end
51
+
30
52
  describe '#step_definitions_path' do
31
53
  it 'returns a default' do
32
54
  subject[:step_definitions_path].must_be_kind_of String
@@ -29,5 +29,15 @@ module Spinach
29
29
  end
30
30
  end
31
31
  end
32
+
33
+ describe "#ordering_id" do
34
+ subject { Feature.new }
35
+
36
+ before { subject.filename = "features/foo/bar.feature" }
37
+
38
+ it 'is the filename' do
39
+ subject.ordering_id.must_equal "features/foo/bar.feature"
40
+ end
41
+ end
32
42
  end
33
43
  end
@@ -108,7 +108,7 @@ describe Spinach::Hooks do
108
108
  subject.on_tag('screenshot') do
109
109
  assertion = true
110
110
  end
111
- subject.run_before_scenario(scenario)
111
+ subject.run_before_scenario(scenario, step_definitions)
112
112
  assertion.wont_equal true
113
113
  end
114
114
  end
@@ -0,0 +1,31 @@
1
+ require_relative '../../test_helper'
2
+
3
+ describe Spinach::Orderers::Default do
4
+ let(:orderer) { Spinach::Orderers::Default.new }
5
+
6
+ describe "#attach_summary" do
7
+ let(:io) { StringIO.new }
8
+
9
+ it 'appends nothing' do
10
+ contents_before_running = io.string.dup
11
+
12
+ orderer.attach_summary(io)
13
+
14
+ io.string.must_equal contents_before_running
15
+ end
16
+ end
17
+
18
+ describe "#order" do
19
+ let(:items) { Array(1..10) }
20
+
21
+ it "doesn't change the order of the items" do
22
+ orderer.order(items).must_equal items
23
+ end
24
+ end
25
+
26
+ describe "#initialize" do
27
+ it "can be provided options without raising an error" do
28
+ Spinach::Orderers::Default.new(seed: "seed")
29
+ end
30
+ end
31
+ end
@@ -0,0 +1,39 @@
1
+ require_relative '../../test_helper'
2
+
3
+ describe Spinach::Orderers::Random do
4
+ let(:orderer) { Spinach::Orderers::Random.new(seed: Spinach.config.seed) }
5
+
6
+ describe "#attach_summary" do
7
+ let(:io) { StringIO.new }
8
+
9
+ it 'appends the seed' do
10
+ orderer.attach_summary(io)
11
+
12
+ io.string.must_match /Randomized\ with\ seed\ #{orderer.seed}/
13
+ end
14
+ end
15
+
16
+ describe "#order" do
17
+ Identifiable = Struct.new(:ordering_id)
18
+
19
+ let(:items) { (1..10).map { |n| Identifiable.new(n.to_s) } }
20
+
21
+ it "randomizes the items" do
22
+ orderer.order(items).wont_equal items
23
+ end
24
+
25
+ it "always randomizes items the same way with the same seed" do
26
+ orderer.order(items).must_equal orderer.order(items)
27
+ end
28
+ end
29
+
30
+ describe "#initialize" do
31
+ it "requires a seed parameter" do
32
+ proc {
33
+ Spinach::Orderers::Random.new
34
+ }.must_raise ArgumentError
35
+
36
+ Spinach::Orderers::Random.new(seed: 4)
37
+ end
38
+ end
39
+ end
@@ -65,7 +65,10 @@ describe Spinach::Runner do
65
65
  config.reporter_options = {backtrace: true}
66
66
  reporter = stub
67
67
  reporter.stubs(:bind)
68
- Spinach::Reporter::Stdout.expects(new: reporter).with(backtrace: true)
68
+ Spinach::Reporter::Stdout.expects(new: reporter).with(
69
+ backtrace: true,
70
+ **runner.default_reporter_options
71
+ )
69
72
  runner.init_reporters
70
73
  end
71
74
  end
@@ -2,5 +2,19 @@ require 'test_helper'
2
2
 
3
3
  module Spinach
4
4
  describe Scenario do
5
+ describe "#ordering_id" do
6
+ let(:feature) { Feature.new }
7
+
8
+ subject { Scenario.new(feature) }
9
+
10
+ before do
11
+ feature.filename = "features/foo/bar.feature"
12
+ subject.lines = Array(4..12)
13
+ end
14
+
15
+ it 'is the filename and starting line number' do
16
+ subject.ordering_id.must_equal "features/foo/bar.feature:4"
17
+ end
18
+ end
5
19
  end
6
20
  end
@@ -13,7 +13,7 @@ end
13
13
 
14
14
  require 'minitest/autorun'
15
15
  require 'minitest/spec'
16
- require 'mocha/mini_test'
16
+ require 'mocha/minitest'
17
17
  require 'ostruct'
18
18
  require 'stringio'
19
19
  require 'pry'
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: spinach
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.10.1
4
+ version: 0.11.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Josep Jaume Rey
@@ -11,7 +11,7 @@ authors:
11
11
  autorequire:
12
12
  bindir: bin
13
13
  cert_chain: []
14
- date: 2018-02-21 00:00:00.000000000 Z
14
+ date: 2020-04-20 00:00:00.000000000 Z
15
15
  dependencies:
16
16
  - !ruby/object:Gem::Dependency
17
17
  name: gherkin-ruby
@@ -75,14 +75,14 @@ dependencies:
75
75
  requirements:
76
76
  - - "~>"
77
77
  - !ruby/object:Gem::Version
78
- version: '1.0'
78
+ version: 1.5.0
79
79
  type: :development
80
80
  prerelease: false
81
81
  version_requirements: !ruby/object:Gem::Requirement
82
82
  requirements:
83
83
  - - "~>"
84
84
  - !ruby/object:Gem::Version
85
- version: '1.0'
85
+ version: 1.5.0
86
86
  - !ruby/object:Gem::Dependency
87
87
  name: sinatra
88
88
  requirement: !ruby/object:Gem::Requirement
@@ -194,10 +194,10 @@ extensions: []
194
194
  extra_rdoc_files: []
195
195
  files:
196
196
  - ".document"
197
+ - ".github/workflows/ci.yml"
197
198
  - ".gitignore"
198
199
  - ".ruby-gemset"
199
200
  - ".ruby-version"
200
- - ".travis.yml"
201
201
  - CHANGELOG.md
202
202
  - Gemfile
203
203
  - Guardfile
@@ -213,6 +213,7 @@ files:
213
213
  - features/fail_fast.feature
214
214
  - features/feature_name_guessing.feature
215
215
  - features/pending_steps.feature
216
+ - features/randomization.feature
216
217
  - features/reporting/customized_reporter.feature
217
218
  - features/reporting/display_run_summary.feature
218
219
  - features/reporting/error_reporting.feature
@@ -230,6 +231,7 @@ files:
230
231
  - features/steps/fail_fast_option.rb
231
232
  - features/steps/feature_name_guessing.rb
232
233
  - features/steps/pending_steps.rb
234
+ - features/steps/randomizing_features_scenarios.rb
233
235
  - features/steps/reporting/display_run_summary.rb
234
236
  - features/steps/reporting/error_reporting.rb
235
237
  - features/steps/reporting/pending_feature_reporting.rb
@@ -263,6 +265,9 @@ files:
263
265
  - lib/spinach/generators/step_generator.rb
264
266
  - lib/spinach/hookable.rb
265
267
  - lib/spinach/hooks.rb
268
+ - lib/spinach/orderers.rb
269
+ - lib/spinach/orderers/default.rb
270
+ - lib/spinach/orderers/random.rb
266
271
  - lib/spinach/parser.rb
267
272
  - lib/spinach/parser/visitor.rb
268
273
  - lib/spinach/reporter.rb
@@ -293,6 +298,8 @@ files:
293
298
  - test/spinach/generators_test.rb
294
299
  - test/spinach/hookable_test.rb
295
300
  - test/spinach/hooks_test.rb
301
+ - test/spinach/orderers/default_test.rb
302
+ - test/spinach/orderers/random_test.rb
296
303
  - test/spinach/parser/visitor_test.rb
297
304
  - test/spinach/parser_test.rb
298
305
  - test/spinach/reporter/failure_file_test.rb
@@ -322,15 +329,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
322
329
  requirements:
323
330
  - - ">="
324
331
  - !ruby/object:Gem::Version
325
- version: '0'
332
+ version: '2.4'
326
333
  required_rubygems_version: !ruby/object:Gem::Requirement
327
334
  requirements:
328
335
  - - ">="
329
336
  - !ruby/object:Gem::Version
330
337
  version: '0'
331
338
  requirements: []
332
- rubyforge_project:
333
- rubygems_version: 2.5.1
339
+ rubygems_version: 3.0.3
334
340
  signing_key:
335
341
  specification_version: 4
336
342
  summary: Spinach is a BDD framework on top of gherkin
@@ -343,6 +349,7 @@ test_files:
343
349
  - features/fail_fast.feature
344
350
  - features/feature_name_guessing.feature
345
351
  - features/pending_steps.feature
352
+ - features/randomization.feature
346
353
  - features/reporting/customized_reporter.feature
347
354
  - features/reporting/display_run_summary.feature
348
355
  - features/reporting/error_reporting.feature
@@ -360,6 +367,7 @@ test_files:
360
367
  - features/steps/fail_fast_option.rb
361
368
  - features/steps/feature_name_guessing.rb
362
369
  - features/steps/pending_steps.rb
370
+ - features/steps/randomizing_features_scenarios.rb
363
371
  - features/steps/reporting/display_run_summary.rb
364
372
  - features/steps/reporting/error_reporting.rb
365
373
  - features/steps/reporting/pending_feature_reporting.rb
@@ -387,6 +395,8 @@ test_files:
387
395
  - test/spinach/generators_test.rb
388
396
  - test/spinach/hookable_test.rb
389
397
  - test/spinach/hooks_test.rb
398
+ - test/spinach/orderers/default_test.rb
399
+ - test/spinach/orderers/random_test.rb
390
400
  - test/spinach/parser/visitor_test.rb
391
401
  - test/spinach/parser_test.rb
392
402
  - test/spinach/reporter/failure_file_test.rb
@@ -1,8 +0,0 @@
1
- rvm:
2
- - 2.1.9
3
- - 2.2.4
4
- - 2.3.1
5
- - jruby-9.1.2.0
6
-
7
- cache: bundler
8
- script: "bundle exec rake"