spinach 0.10.1 → 0.12.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (48) hide show
  1. checksums.yaml +5 -5
  2. data/.github/workflows/ci.yml +30 -0
  3. data/.ruby-version +1 -1
  4. data/.tool-versions +1 -0
  5. data/Gemfile +0 -1
  6. data/README.markdown +35 -38
  7. data/Rakefile +1 -1
  8. data/bin/spinach +1 -1
  9. data/features/feature_hooks_and_tags.feature +22 -0
  10. data/features/randomization.feature +16 -0
  11. data/features/reporting/display_run_summary.feature +19 -3
  12. data/features/steps/automatic_feature_generation.rb +1 -1
  13. data/features/steps/feature_hooks_and_tags.rb +55 -0
  14. data/features/steps/randomizing_features_scenarios.rb +68 -0
  15. data/features/steps/reporting/display_run_summary.rb +29 -1
  16. data/features/support/env.rb +0 -13
  17. data/features/support/spinach_runner.rb +11 -0
  18. data/lib/spinach/cli.rb +29 -2
  19. data/lib/spinach/config.rb +25 -0
  20. data/lib/spinach/feature.rb +8 -0
  21. data/lib/spinach/orderers/default.rb +25 -0
  22. data/lib/spinach/orderers/random.rb +35 -0
  23. data/lib/spinach/orderers.rb +2 -0
  24. data/lib/spinach/reporter/progress.rb +1 -1
  25. data/lib/spinach/reporter/reporting.rb +3 -1
  26. data/lib/spinach/reporter/stdout.rb +1 -1
  27. data/lib/spinach/reporter.rb +1 -0
  28. data/lib/spinach/runner/feature_runner.rb +7 -3
  29. data/lib/spinach/runner.rb +40 -17
  30. data/lib/spinach/scenario.rb +10 -0
  31. data/lib/spinach/tags_matcher.rb +11 -2
  32. data/lib/spinach/version.rb +1 -1
  33. data/lib/spinach.rb +1 -0
  34. data/spinach.gemspec +4 -5
  35. data/test/spinach/cli_test.rb +39 -5
  36. data/test/spinach/config_test.rb +22 -0
  37. data/test/spinach/dsl_test.rb +1 -1
  38. data/test/spinach/feature_test.rb +10 -0
  39. data/test/spinach/generators/feature_generator_test.rb +1 -1
  40. data/test/spinach/hooks_test.rb +1 -1
  41. data/test/spinach/orderers/default_test.rb +31 -0
  42. data/test/spinach/orderers/random_test.rb +39 -0
  43. data/test/spinach/reporter_test.rb +2 -2
  44. data/test/spinach/runner_test.rb +28 -8
  45. data/test/spinach/scenario_test.rb +14 -0
  46. data/test/test_helper.rb +1 -14
  47. metadata +26 -40
  48. data/.travis.yml +0 -8
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
- SHA1:
3
- metadata.gz: 1f75603856e6e68fb9f7eb16a287062659034e02
4
- data.tar.gz: a6ea70aaa3ef522f10e6b5c729671c94c84bc67c
2
+ SHA256:
3
+ metadata.gz: 13ae199ec9413a41d2f1ea007433f4ca9acf8750811468e320927c55cc6ae80d
4
+ data.tar.gz: 37383a9a6e243a01d113cbe64139ce7119af7c571dcb12589846b8737115ac9e
5
5
  SHA512:
6
- metadata.gz: c5f580a7a0bc2c9f4ce8901b78f1f63da2a72d14f8233ade3ebd423f6bf0989441b318970266610422b6e53de854fd6cf1ce868f53e80064f6606396d9c39c3f
7
- data.tar.gz: 349c9aa0672871576ea8ff8f86944c24816048ccbee2ca3a360b446feed8d6ede0ab2671a8db2505ea4cfd80a0f3e5e66f55578e88d69db215981d37970c06a3
6
+ metadata.gz: 16007d35b6d52765c78a9e82e7f08e484d6d8f6ddc8ff4de765472ec11c7e236d6dd92de7cb5e89f8d8bda7ddc042fddbc2480ba92b00793784ca5d2d22c0e12
7
+ data.tar.gz: ff6aad6deac02d61cf3f932afacf62bedb0f16aaa07e415ea1a675c7ac79e815d030ebb86f039e4d726bd5481226f12326c9d9060d0c4f999c820e5226a1a118
@@ -0,0 +1,30 @@
1
+ name: "Tests"
2
+
3
+ on:
4
+ push:
5
+ branches:
6
+ - master
7
+ pull_request:
8
+ branches:
9
+ - "*"
10
+
11
+ env:
12
+ CI: "true"
13
+
14
+ jobs:
15
+ main:
16
+ name: Tests
17
+ runs-on: ubuntu-latest
18
+ strategy:
19
+ matrix:
20
+ ruby: [2.4, 2.5, 2.6, 2.7, "3.0", 3.1, 3.2, jruby]
21
+ fail-fast: false
22
+ steps:
23
+ - uses: actions/checkout@v3
24
+ with:
25
+ fetch-depth: 1
26
+ - uses: ruby/setup-ruby@v1
27
+ with:
28
+ ruby-version: ${{ matrix.ruby }}
29
+ bundler-cache: true
30
+ - run: bundle exec rake
data/.ruby-version CHANGED
@@ -1 +1 @@
1
- 2.3.1
1
+ 3.2.0
data/.tool-versions ADDED
@@ -0,0 +1 @@
1
+ ruby 3.2.1
data/Gemfile CHANGED
@@ -3,7 +3,6 @@ source 'http://rubygems.org'
3
3
  # Specify your gem's dependencies in spinach.gemspec
4
4
  gemspec
5
5
 
6
- gem 'coveralls', require: false
7
6
  gem 'pry-byebug', platforms: [:ruby]
8
7
 
9
8
  group :docs do
data/README.markdown CHANGED
@@ -1,8 +1,7 @@
1
1
  # Spinach - BDD framework on top of Gherkin
2
- [![Gem Version](https://badge.fury.io/rb/spinach.png)](http://badge.fury.io/rb/spinach)
3
- [![Build Status](https://secure.travis-ci.org/codegram/spinach.png)](http://travis-ci.org/codegram/spinach)
4
- [![Dependency Status](https://gemnasium.com/codegram/spinach.png)](http://gemnasium.com/codegram/spinach)
5
- [![Coverage Status](https://coveralls.io/repos/codegram/spinach/badge.png?branch=master)](https://coveralls.io/r/codegram/spinach)
2
+
3
+ [![Gem Version](https://badge.fury.io/rb/spinach.svg)](https://badge.fury.io/rb/spinach)
4
+ ![Tests](https://github.com/codegram/spinach/workflows/Tests/badge.svg)
6
5
 
7
6
  Spinach is a high-level BDD framework that leverages the expressive
8
7
  [Gherkin language][gherkin] (used by [Cucumber][cucumber]) to help you define
@@ -10,13 +9,13 @@ executable specifications of your application or library's acceptance criteria.
10
9
 
11
10
  Conceived as an alternative to Cucumber, here are some of its design goals:
12
11
 
13
- * Step maintainability: since features map to their own classes, their steps are
12
+ - Step maintainability: since features map to their own classes, their steps are
14
13
  just methods of that class. This encourages step encapsulation.
15
14
 
16
- * Step reusability: In case you want to reuse steps across features, you can
15
+ - Step reusability: In case you want to reuse steps across features, you can
17
16
  always wrap those in plain ol' Ruby modules.
18
17
 
19
- Spinach is tested against **2.0, 2.1, 2.2 and 2.3** as well as **JRuby 9000**.
18
+ Spinach is tested against Ruby MRI **2.4, 2.5, 2.6, 2.7, 3.0, 3.1, 3.2** as well as **JRuby **.
20
19
 
21
20
  ## Getting started
22
21
 
@@ -184,7 +183,7 @@ end
184
183
  ## Audit
185
184
 
186
185
  Over time, the definitions of your features will change. When you add, remove
187
- or change steps in the feature files, you can easily audit your existing step
186
+ or change steps in the feature files, you can easily audit your existing step
188
187
  files with:
189
188
 
190
189
  ```shell
@@ -198,12 +197,11 @@ This does not modify the step files, so you will need to paste the boilerplate
198
197
  into the appropriate places. If a new feature file is detected, you will be
199
198
  asked to run `spinach --generate` beforehand.
200
199
 
201
- **Important**: If auditing individual files, common steps (as above) may be
202
- reported as unused when they are actually used in a feature file that is not
200
+ **Important**: If auditing individual files, common steps (as above) may be
201
+ reported as unused when they are actually used in a feature file that is not
203
202
  currently being audited. To avoid this, run the audit with no arguments to
204
203
  audit all step files simultaneously.
205
204
 
206
-
207
205
  ## Tags
208
206
 
209
207
  Feature and Scenarios can be marked with tags in the form: `@tag`. Tags can be
@@ -238,19 +236,19 @@ Feature: So something great
238
236
  Scenario: Ensure no regression on this
239
237
  ```
240
238
 
241
- Then you can run all Scenarios in your suite related to `@feat-1` using:
239
+ Then you can run all Scenarios in your suite tagged `@feat-1` using:
242
240
 
243
241
  ```shell
244
242
  $ spinach --tags @feat-1
245
243
  ```
246
244
 
247
- Or only Scenarios related to `@feat-1` and `@bug-12` using:
245
+ Or only Scenarios tagged either `@feat-1` or `@bug-12` using:
248
246
 
249
247
  ```shell
250
248
  $ spinach --tags @feat-1,@bug-12
251
249
  ```
252
250
 
253
- Or only Scenarios related to `@feat-1` excluding `@bug-12` using:
251
+ Or only Scenarios tagged `@feat-1` that aren't tagged `@bug-12` using:
254
252
 
255
253
  ```shell
256
254
  $ spinach --tags @feat-1,~@bug-12
@@ -292,7 +290,7 @@ Full hook documentation is here:
292
290
 
293
291
  ## Local Before and After Hooks
294
292
 
295
- Sometimes it feels awkward to add steps into feature file just because you need to do some test setup and cleanup. And it is equally awkward to add a global hooks for this purpose. For example, if you want to add a session timeout feature, to do so, you want to set the session timeout time to 1 second just for this feature, and put the normal timeout back after this feature. It doesn't make sense to add two steps in the feature file just to change the session timeout value. In this scenario, a ```before``` and ```after``` blocks are perfect for this kind of tasks. Below is an example implementation:
293
+ Sometimes it feels awkward to add steps into feature file just because you need to do some test setup and cleanup. And it is equally awkward to add a global hooks for this purpose. For example, if you want to add a session timeout feature, to do so, you want to set the session timeout time to 1 second just for this feature, and put the normal timeout back after this feature. It doesn't make sense to add two steps in the feature file just to change the session timeout value. In this scenario, a `before` and `after` blocks are perfect for this kind of tasks. Below is an example implementation:
296
294
 
297
295
  ```ruby
298
296
  class Spinach::Features::SessionTimeout < Spinach::FeatureSteps
@@ -329,8 +327,8 @@ When no reporter is specified, `stdout` will be used by default.
329
327
 
330
328
  Other reporters:
331
329
 
332
- * For a console reporter with no colors, try [spinach-console-reporter][spinach-console-reporter] (to be used with Jenkins)
333
- * For a rerun reporter, try [spinach-rerun-reporter][spinach-rerun-reporter] (writes failed scenarios in a file)
330
+ - For a console reporter with no colors, try [spinach-console-reporter][spinach-console-reporter] (to be used with Jenkins)
331
+ - For a rerun reporter, try [spinach-rerun-reporter][spinach-rerun-reporter] (writes failed scenarios in a file)
334
332
 
335
333
  ## Wanna use it with Rails 3?
336
334
 
@@ -342,46 +340,45 @@ Check out our [spinach-sinatra demo](https://github.com/codegram/spinach-sinatra
342
340
 
343
341
  ## Resources
344
342
 
345
- * [Landing page](http://codegram.github.com/spinach)
346
- * [Slides](http://codegram.github.com/spinach-presentation)
347
- * [Blog post](http://blog.codegram.com/2011/10/how-to-achieve-more-clean-encapsulated-modular-step-definitions-with-spinach)
348
- * [API Documentation](http://rubydoc.info/github/codegram/spinach/master/frames)
349
- * [Google group](https://groups.google.com/forum/#!forum/spinach_bdd)
343
+ - [Landing page](http://codegram.github.com/spinach)
344
+ - [Slides](http://codegram.github.com/spinach-presentation)
345
+ - [Blog post](http://blog.codegram.com/2011/10/how-to-achieve-more-clean-encapsulated-modular-step-definitions-with-spinach)
346
+ - [API Documentation](http://rubydoc.info/github/codegram/spinach/master/frames)
347
+ - [Google group](https://groups.google.com/forum/#!forum/spinach_bdd)
350
348
 
351
349
  ### Related gems
352
350
 
353
- * [guard-spinach](http://github.com/codegram/guard-spinach)
354
- * [spinach-rails](http://github.com/codegram/spinach-rails)
355
- * [spinach-console-reporter][spinach-console-reporter] (to be used with Jenkins)
356
- * [spinach-rerun-reporter][spinach-rerun-reporter] (writes failed scenarios in a file)
357
- * [spring-commands-spinach](https://github.com/jvanbaarsen/spring-commands-spinach) (to be used with spring)
351
+ - [guard-spinach](http://github.com/codegram/guard-spinach)
352
+ - [spinach-rails](http://github.com/codegram/spinach-rails)
353
+ - [spinach-console-reporter][spinach-console-reporter] (to be used with Jenkins)
354
+ - [spinach-rerun-reporter][spinach-rerun-reporter] (writes failed scenarios in a file)
355
+ - [spring-commands-spinach](https://github.com/jvanbaarsen/spring-commands-spinach) (to be used with spring)
358
356
 
359
357
  ### Demos
360
358
 
361
- * [spinach rails demo](https://github.com/codegram/spinach-rails-demo)
362
- * [spinach sinatra demo](https://github.com/codegram/spinach-sinatra-demo)
363
- * [simple todo Rails app](https://github.com/codegram/tasca-spinach-demo)
359
+ - [spinach rails demo](https://github.com/codegram/spinach-rails-demo)
360
+ - [spinach sinatra demo](https://github.com/codegram/spinach-sinatra-demo)
361
+ - [simple todo Rails app](https://github.com/codegram/tasca-spinach-demo)
364
362
 
365
363
  ## Contributing
366
364
 
367
- * [List of spinach contributors](https://github.com/codegram/spinach/contributors)
365
+ - [List of spinach contributors](https://github.com/codegram/spinach/contributors)
368
366
 
369
367
  You can easily contribute to Spinach. Its codebase is simple and
370
368
  [extensively documented][documentation].
371
369
 
372
- * Fork the project.
373
- * Make your feature addition or bug fix.
374
- * Add specs for it. This is important so we don't break it in a future
370
+ - Fork the project.
371
+ - Make your feature addition or bug fix.
372
+ - Add specs for it. This is important so we don't break it in a future
375
373
  version unintentionally.
376
- * Commit, do not mess with rakefile, version, or history.
374
+ - Commit, do not mess with rakefile, version, or history.
377
375
  If you want to have your own version, that is fine but bump version
378
376
  in a commit by itself I can ignore when I pull.
379
- * Send me a pull request. Bonus points for topic branches.
377
+ - Send me a pull request. Bonus points for topic branches.
380
378
 
381
379
  ## License
382
380
 
383
- MIT (Expat) License. Copyright 2011-2016 [Codegram Technologies](http://codegram.com)
384
-
381
+ MIT (Expat) License. Copyright 2011-2023 [Codegram Technologies](http://codegram.com)
385
382
 
386
383
  [gherkin]: http://github.com/codegram/gherkin-ruby
387
384
  [cucumber]: http://github.com/cucumber/cucumber
data/Rakefile CHANGED
@@ -11,7 +11,7 @@ end
11
11
 
12
12
  desc 'Run spinach features'
13
13
  task :spinach do
14
- exec "bin/spinach"
14
+ exec "bin/spinach --rand"
15
15
  end
16
16
 
17
17
  task :default => [:test, :spinach]
data/bin/spinach CHANGED
@@ -1,7 +1,7 @@
1
1
  #!/usr/bin/env ruby
2
2
 
3
3
  begin
4
- require "bundler/setup"
4
+ require "bundler/setup"
5
5
  rescue LoadError
6
6
  end
7
7
 
@@ -0,0 +1,22 @@
1
+ Feature: Feature Hooks and Tags
2
+ In order to run only the appropriate setup and teardown code
3
+ As a developer
4
+ I want spinach to only run feature hooks if those features would be run under the tags I provided
5
+
6
+ Scenario: No tags specified
7
+ Given I have a tagged feature with an untagged scenario
8
+ And I have an untagged feature with a tagged scenario
9
+ When I don't specify tags
10
+ Then all the feature hooks should have run
11
+
12
+ Scenario: Tags specified
13
+ Given I have a tagged feature with an untagged scenario
14
+ And I have an untagged feature with a tagged scenario
15
+ When I specify a tag the features and scenarios are tagged with
16
+ Then all the feature hooks should have run
17
+
18
+ Scenario: Tags excluded
19
+ Given I have a tagged feature with an untagged scenario
20
+ And I have an untagged feature with a tagged scenario
21
+ When I exclude a tag the features and scenarios are tagged with
22
+ Then no feature hooks should have run
@@ -0,0 +1,16 @@
1
+ Feature: Randomizing Features & Scenarios
2
+ In order to ensure my tests aren't dependent
3
+ As a developer
4
+ I want spinach to randomize features and scenarios (but not steps)
5
+
6
+ Scenario: Randomizing the run without specifying a seed
7
+ Given I have 2 features with 2 scenarios each
8
+ When I randomize the run without specifying a seed
9
+ Then The features and scenarios are run
10
+ And The runner output shows a seed
11
+
12
+ Scenario: Specifying the seed
13
+ Given I have 2 features with 2 scenarios each
14
+ When I specify the seed for the run
15
+ Then The features and scenarios are run in a different order
16
+ And The runner output shows the seed
@@ -1,9 +1,25 @@
1
1
  Feature: Display run summary
2
2
  As a developer
3
3
  I want spinach to display a summary of steps statuses
4
- So I can easyly know general features status
4
+ So I can easily know general features status
5
5
 
6
- Scenario: Display run summary at the end of features run
6
+ Scenario: Display run summary at the end of features run without randomization
7
7
  Given I have a feature that has some successful, undefined, failed and error steps
8
- When I run it
8
+
9
+ When I run it without randomization
10
+ Then I should see a summary with steps status information
11
+ And I shouldn't see a randomization seed
12
+
13
+ Scenario: Display run summary at the end of features run with randomization
14
+ Given I have a feature that has some successful, undefined, failed and error steps
15
+
16
+ When I run it with randomization
17
+ Then I should see a summary with steps status information
18
+ And I should see a randomization seed
19
+
20
+ Scenario: Display run summary at the end of features run with a randomization seed
21
+ Given I have a feature that has some successful, undefined, failed and error steps
22
+
23
+ When I run it with a specific randomization seed
9
24
  Then I should see a summary with steps status information
25
+ And I should see that specific randomization seed
@@ -20,7 +20,7 @@ Feature: Cheezburger can I has
20
20
  Then 'I a feature should exist named "features/steps/cheezburger_can_i_has.rb"' do
21
21
  in_current_dir do
22
22
  @file = 'features/steps/cheezburger_can_i_has.rb'
23
- File.exists?(@file).must_equal true
23
+ File.exist?(@file).must_equal true
24
24
  end
25
25
  end
26
26
 
@@ -0,0 +1,55 @@
1
+ class Spinach::Features::FeatureHooksAndTags < Spinach::FeatureSteps
2
+ include Integration::SpinachRunner
3
+
4
+ step 'I have a tagged feature with an untagged scenario' do
5
+ write_file 'features/a.feature', <<-FEATURE
6
+ @tag
7
+ Feature: A
8
+ Scenario: A1
9
+ Then a1
10
+ FEATURE
11
+
12
+ write_file 'features/steps/a.rb', <<-STEPS
13
+ class Spinach::Features::A < Spinach::FeatureSteps
14
+ step 'a1' do; end
15
+ end
16
+ STEPS
17
+ end
18
+
19
+ step 'I have an untagged feature with a tagged scenario' do
20
+ write_file 'features/b.feature', <<-FEATURE
21
+ Feature: B
22
+ @tag
23
+ Scenario: B1
24
+ Then b1
25
+ FEATURE
26
+
27
+ write_file 'features/steps/b.rb', <<-STEPS
28
+ class Spinach::Features::B < Spinach::FeatureSteps
29
+ step 'b1' do; end
30
+ end
31
+ STEPS
32
+ end
33
+
34
+ step "I don't specify tags" do
35
+ run_spinach
36
+ end
37
+
38
+ step 'I specify a tag the features and scenarios are tagged with' do
39
+ run_spinach({append: "--tags @tag"})
40
+ end
41
+
42
+ step 'I exclude a tag the features and scenarios are tagged with' do
43
+ run_spinach({append: "--tags ~@tag"})
44
+ end
45
+
46
+ step 'all the feature hooks should have run' do
47
+ @stdout.must_match("Feature: A")
48
+ @stdout.must_match("Feature: B")
49
+ end
50
+
51
+ step 'no feature hooks should have run' do
52
+ @stdout.wont_match("Feature: A")
53
+ @stdout.wont_match("Feature: B")
54
+ end
55
+ end
@@ -0,0 +1,68 @@
1
+ class Spinach::Features::RandomizingFeaturesScenarios < Spinach::FeatureSteps
2
+ include Integration::SpinachRunner
3
+
4
+ step 'I have 2 features with 2 scenarios each' do
5
+ write_file 'features/success_a.feature', <<-FEATURE
6
+ Feature: Success A
7
+ Scenario: A1
8
+ Then a1
9
+ Scenario: A2
10
+ Then a2
11
+ FEATURE
12
+
13
+ write_file 'features/steps/success_a.rb', <<-STEPS
14
+ class Spinach::Features::SuccessA < Spinach::FeatureSteps
15
+ step 'a1' do; end
16
+ step 'a2' do; end
17
+ end
18
+ STEPS
19
+
20
+ write_file 'features/success_b.feature', <<-FEATURE
21
+ Feature: Success B
22
+ Scenario: B1
23
+ Then b1
24
+ Scenario: B2
25
+ Then b2
26
+ FEATURE
27
+
28
+ write_file 'features/steps/success_b.rb', <<-STEPS
29
+ class Spinach::Features::SuccessB < Spinach::FeatureSteps
30
+ step 'b1' do; end
31
+ step 'b2' do; end
32
+ end
33
+ STEPS
34
+ end
35
+
36
+ step 'I randomize the run without specifying a seed' do
37
+ run_spinach({append: "--rand"})
38
+ end
39
+
40
+ step 'I specify the seed for the run' do
41
+ # Reverse order (A2 A1 B2 B1) is the only way I can show that
42
+ # scenarios and features are randomized by the seed when the
43
+ # example has 2 features each with 2 scenarios. I tried seeds
44
+ # until I found one that ordered the test in that order.
45
+ @seed = 1
46
+
47
+ run_spinach({append: "--seed #{@seed}"})
48
+ end
49
+
50
+ step 'The features and scenarios are run' do
51
+ @stdout.must_include("A1")
52
+ @stdout.must_include("A2")
53
+ @stdout.must_include("B1")
54
+ @stdout.must_include("B2")
55
+ end
56
+
57
+ step 'The features and scenarios are run in a different order' do
58
+ @stdout.must_match(/B2.*B1.*A2.*A1/m)
59
+ end
60
+
61
+ step 'The runner output shows a seed' do
62
+ @stdout.must_match(/^Randomized with seed \d*$/)
63
+ end
64
+
65
+ step 'The runner output shows the seed' do
66
+ @stdout.must_match(/^Randomized with seed #{@seed}$/)
67
+ end
68
+ end
@@ -62,7 +62,7 @@ Feature: A test feature
62
62
  @feature = "features/test_feature.feature"
63
63
  end
64
64
 
65
- When "I run it" do
65
+ When "I run it without randomization" do
66
66
  run_feature @feature
67
67
  end
68
68
 
@@ -71,4 +71,32 @@ Feature: A test feature
71
71
  /Summary:.*4.*Successful.*1.*Undefined.*1.*Failed.*1.*Error/
72
72
  )
73
73
  end
74
+
75
+ And "I shouldn't see a randomization seed" do
76
+ @stdout.wont_match(
77
+ /Randomized\ with\ seed\ \d+/
78
+ )
79
+ end
80
+
81
+ When "I run it with randomization" do
82
+ run_feature @feature, {append: "--rand"}
83
+ end
84
+
85
+ And "I should see a randomization seed" do
86
+ @stdout.must_match(
87
+ /Randomized\ with\ seed\ \d+/
88
+ )
89
+ end
90
+
91
+ When "I run it with a specific randomization seed" do
92
+ @seed = rand(0xFFFF)
93
+
94
+ run_feature @feature, {append: "--seed #{@seed}"}
95
+ end
96
+
97
+ And "I should see that specific randomization seed" do
98
+ @stdout.must_match(
99
+ /Randomized\ with\ seed\ #{@seed}/
100
+ )
101
+ end
74
102
  end
@@ -1,19 +1,6 @@
1
1
  require 'minitest/autorun'
2
2
  require 'minitest/spec'
3
3
  require_relative 'filesystem'
4
- require 'simplecov'
5
-
6
- if ENV['CI'] && !defined?(Rubinius)
7
- require 'coveralls'
8
- SimpleCov.formatter = Coveralls::SimpleCov::Formatter
9
- require 'simplecov'
10
-
11
- SimpleCov.start do
12
- add_filter '/test/'
13
- add_filter '/features/'
14
- end
15
- end
16
-
17
4
 
18
5
  Spinach.hooks.after_scenario do |scenario|
19
6
  FileUtils.rm_rf(Filesystem.dirs)
@@ -23,6 +23,17 @@ module Integration
23
23
  run "#{ruby} #{spinach} #{feature} #{options[:append]}", options[:env]
24
24
  end
25
25
 
26
+ def run_spinach(options = {})
27
+ options[:framework] ||= :minitest
28
+
29
+ use_minitest if options[:framework] == :minitest
30
+ use_rspec if options[:framework] == :rspec
31
+
32
+ spinach = File.expand_path("bin/spinach")
33
+
34
+ run "#{ruby} #{spinach} #{options[:append]}"
35
+ end
36
+
26
37
  def ruby
27
38
  return @ruby if defined?(@ruby)
28
39
 
data/lib/spinach/cli.rb CHANGED
@@ -56,7 +56,7 @@ module Spinach
56
56
 
57
57
  @args.each do |arg|
58
58
  if arg.match(/\.feature/)
59
- if File.exists? arg.gsub(/:\d*/, '')
59
+ if File.exist? arg.gsub(/:\d*/, '')
60
60
  files_to_run << arg
61
61
  else
62
62
  fail! "#{arg} could not be found"
@@ -131,6 +131,16 @@ module Spinach
131
131
  config[:reporter_classes] = names
132
132
  end
133
133
 
134
+ opts.on('--rand', "Randomize the order of features and scenarios") do
135
+ config[:orderer_class] = orderer_class(:random)
136
+ end
137
+
138
+ opts.on('--seed SEED', Integer,
139
+ "Provide a seed for randomizing the order of features and scenarios") do |seed|
140
+ config[:orderer_class] = orderer_class(:random)
141
+ config[:seed] = seed
142
+ end
143
+
134
144
  opts.on_tail('--fail-fast',
135
145
  'Terminate the suite run on the first failure') do |class_name|
136
146
  config[:fail_fast] = true
@@ -164,7 +174,7 @@ and obsolete steps") do
164
174
  # Builds the class name to use an output reporter.
165
175
  #
166
176
  # @param [String] klass
167
- # The class name fo the reporter.
177
+ # The class name of the reporter.
168
178
  #
169
179
  # @return [String]
170
180
  # The full name of the reporter class.
@@ -177,5 +187,22 @@ and obsolete steps") do
177
187
  def reporter_class(klass)
178
188
  "Spinach::Reporter::" + Spinach::Support.camelize(klass)
179
189
  end
190
+
191
+ # Builds the class to use an orderer.
192
+ #
193
+ # @param [String] klass
194
+ # The class name of the orderer.
195
+ #
196
+ # @return [String]
197
+ # The full name of the orderer class.
198
+ #
199
+ # @example
200
+ # orderer_class('random')
201
+ # # => Spinach::Orderers::Random
202
+ #
203
+ # @api private
204
+ def orderer_class(klass)
205
+ "Spinach::Orderers::" + Spinach::Support.camelize(klass)
206
+ end
180
207
  end
181
208
  end
@@ -33,6 +33,8 @@ module Spinach
33
33
  :save_and_open_page_on_failure,
34
34
  :reporter_classes,
35
35
  :reporter_options,
36
+ :orderer_class,
37
+ :seed,
36
38
  :fail_fast,
37
39
  :audit
38
40
 
@@ -66,6 +68,29 @@ module Spinach
66
68
  @reporter_options || {}
67
69
  end
68
70
 
71
+ # The "orderer class" holds the orderer class name
72
+ # Defaults to Spinach::Orderers::Default
73
+ #
74
+ # @return [orderer object]
75
+ # The orderer that responds to specific messages.
76
+ #
77
+ # @api public
78
+ def orderer_class
79
+ @orderer_class || "Spinach::Orderers::Default"
80
+ end
81
+
82
+ # A randomization seed. This is what spinach uses for test run
83
+ # randomization, so if you call `Kernel.srand(Spinach.config.seed)`
84
+ # in your support environment file, not only will the test run
85
+ # order be guaranteed to be stable under a specific seed, all
86
+ # the Ruby-generated random numbers produced during your test
87
+ # run will also be stable under that seed.
88
+ #
89
+ # @api public
90
+ def seed
91
+ @seed ||= rand(0xFFFF)
92
+ end
93
+
69
94
  # The "step definitions path" holds the place where your feature step
70
95
  # classes will be searched for. Defaults to '#{features_path}/steps'
71
96
  #
@@ -24,6 +24,14 @@ module Spinach
24
24
  lines_to_run.empty?
25
25
  end
26
26
 
27
+ # Identifier used by orderers.
28
+ #
29
+ # Needs to involve the relative file path so that the ordering
30
+ # a seed generates is stable across both runs and machines.
31
+ #
32
+ # @api public
33
+ alias ordering_id filename
34
+
27
35
  # Run the provided code for every step
28
36
  def each_step
29
37
  scenarios.each { |scenario| scenario.steps.each { |step| yield step } }
@@ -0,0 +1,25 @@
1
+ module Spinach
2
+ module Orderers
3
+ class Default
4
+ def initialize(**options); end
5
+
6
+ # Appends any necessary report output (by default does nothing).
7
+ #
8
+ # @param [IO] io
9
+ # Output buffer for report.
10
+ #
11
+ # @api public
12
+ def attach_summary(io); end
13
+
14
+ # Returns a reordered version of the provided array
15
+ #
16
+ # @param [Array] items
17
+ # Items to order
18
+ #
19
+ # @api public
20
+ def order(items)
21
+ items
22
+ end
23
+ end
24
+ end
25
+ end