spinach 0.10.1 → 0.12.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +5 -5
- data/.github/workflows/ci.yml +30 -0
- data/.ruby-version +1 -1
- data/.tool-versions +1 -0
- data/Gemfile +0 -1
- data/README.markdown +35 -38
- data/Rakefile +1 -1
- data/bin/spinach +1 -1
- data/features/feature_hooks_and_tags.feature +22 -0
- data/features/randomization.feature +16 -0
- data/features/reporting/display_run_summary.feature +19 -3
- data/features/steps/automatic_feature_generation.rb +1 -1
- data/features/steps/feature_hooks_and_tags.rb +55 -0
- data/features/steps/randomizing_features_scenarios.rb +68 -0
- data/features/steps/reporting/display_run_summary.rb +29 -1
- data/features/support/env.rb +0 -13
- data/features/support/spinach_runner.rb +11 -0
- data/lib/spinach/cli.rb +29 -2
- data/lib/spinach/config.rb +25 -0
- data/lib/spinach/feature.rb +8 -0
- data/lib/spinach/orderers/default.rb +25 -0
- data/lib/spinach/orderers/random.rb +35 -0
- data/lib/spinach/orderers.rb +2 -0
- data/lib/spinach/reporter/progress.rb +1 -1
- data/lib/spinach/reporter/reporting.rb +3 -1
- data/lib/spinach/reporter/stdout.rb +1 -1
- data/lib/spinach/reporter.rb +1 -0
- data/lib/spinach/runner/feature_runner.rb +7 -3
- data/lib/spinach/runner.rb +40 -17
- data/lib/spinach/scenario.rb +10 -0
- data/lib/spinach/tags_matcher.rb +11 -2
- data/lib/spinach/version.rb +1 -1
- data/lib/spinach.rb +1 -0
- data/spinach.gemspec +4 -5
- data/test/spinach/cli_test.rb +39 -5
- data/test/spinach/config_test.rb +22 -0
- data/test/spinach/dsl_test.rb +1 -1
- data/test/spinach/feature_test.rb +10 -0
- data/test/spinach/generators/feature_generator_test.rb +1 -1
- data/test/spinach/hooks_test.rb +1 -1
- data/test/spinach/orderers/default_test.rb +31 -0
- data/test/spinach/orderers/random_test.rb +39 -0
- data/test/spinach/reporter_test.rb +2 -2
- data/test/spinach/runner_test.rb +28 -8
- data/test/spinach/scenario_test.rb +14 -0
- data/test/test_helper.rb +1 -14
- metadata +26 -40
- data/.travis.yml +0 -8
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
|
-
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: 13ae199ec9413a41d2f1ea007433f4ca9acf8750811468e320927c55cc6ae80d
|
4
|
+
data.tar.gz: 37383a9a6e243a01d113cbe64139ce7119af7c571dcb12589846b8737115ac9e
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 16007d35b6d52765c78a9e82e7f08e484d6d8f6ddc8ff4de765472ec11c7e236d6dd92de7cb5e89f8d8bda7ddc042fddbc2480ba92b00793784ca5d2d22c0e12
|
7
|
+
data.tar.gz: ff6aad6deac02d61cf3f932afacf62bedb0f16aaa07e415ea1a675c7ac79e815d030ebb86f039e4d726bd5481226f12326c9d9060d0c4f999c820e5226a1a118
|
@@ -0,0 +1,30 @@
|
|
1
|
+
name: "Tests"
|
2
|
+
|
3
|
+
on:
|
4
|
+
push:
|
5
|
+
branches:
|
6
|
+
- master
|
7
|
+
pull_request:
|
8
|
+
branches:
|
9
|
+
- "*"
|
10
|
+
|
11
|
+
env:
|
12
|
+
CI: "true"
|
13
|
+
|
14
|
+
jobs:
|
15
|
+
main:
|
16
|
+
name: Tests
|
17
|
+
runs-on: ubuntu-latest
|
18
|
+
strategy:
|
19
|
+
matrix:
|
20
|
+
ruby: [2.4, 2.5, 2.6, 2.7, "3.0", 3.1, 3.2, jruby]
|
21
|
+
fail-fast: false
|
22
|
+
steps:
|
23
|
+
- uses: actions/checkout@v3
|
24
|
+
with:
|
25
|
+
fetch-depth: 1
|
26
|
+
- uses: ruby/setup-ruby@v1
|
27
|
+
with:
|
28
|
+
ruby-version: ${{ matrix.ruby }}
|
29
|
+
bundler-cache: true
|
30
|
+
- run: bundle exec rake
|
data/.ruby-version
CHANGED
@@ -1 +1 @@
|
|
1
|
-
2.
|
1
|
+
3.2.0
|
data/.tool-versions
ADDED
@@ -0,0 +1 @@
|
|
1
|
+
ruby 3.2.1
|
data/Gemfile
CHANGED
data/README.markdown
CHANGED
@@ -1,8 +1,7 @@
|
|
1
1
|
# Spinach - BDD framework on top of Gherkin
|
2
|
-
|
3
|
-
[![
|
4
|
-
|
5
|
-
[![Coverage Status](https://coveralls.io/repos/codegram/spinach/badge.png?branch=master)](https://coveralls.io/r/codegram/spinach)
|
2
|
+
|
3
|
+
[![Gem Version](https://badge.fury.io/rb/spinach.svg)](https://badge.fury.io/rb/spinach)
|
4
|
+
![Tests](https://github.com/codegram/spinach/workflows/Tests/badge.svg)
|
6
5
|
|
7
6
|
Spinach is a high-level BDD framework that leverages the expressive
|
8
7
|
[Gherkin language][gherkin] (used by [Cucumber][cucumber]) to help you define
|
@@ -10,13 +9,13 @@ executable specifications of your application or library's acceptance criteria.
|
|
10
9
|
|
11
10
|
Conceived as an alternative to Cucumber, here are some of its design goals:
|
12
11
|
|
13
|
-
|
12
|
+
- Step maintainability: since features map to their own classes, their steps are
|
14
13
|
just methods of that class. This encourages step encapsulation.
|
15
14
|
|
16
|
-
|
15
|
+
- Step reusability: In case you want to reuse steps across features, you can
|
17
16
|
always wrap those in plain ol' Ruby modules.
|
18
17
|
|
19
|
-
Spinach is tested against **2.
|
18
|
+
Spinach is tested against Ruby MRI **2.4, 2.5, 2.6, 2.7, 3.0, 3.1, 3.2** as well as **JRuby **.
|
20
19
|
|
21
20
|
## Getting started
|
22
21
|
|
@@ -184,7 +183,7 @@ end
|
|
184
183
|
## Audit
|
185
184
|
|
186
185
|
Over time, the definitions of your features will change. When you add, remove
|
187
|
-
or change steps in the feature files, you can easily audit your existing step
|
186
|
+
or change steps in the feature files, you can easily audit your existing step
|
188
187
|
files with:
|
189
188
|
|
190
189
|
```shell
|
@@ -198,12 +197,11 @@ This does not modify the step files, so you will need to paste the boilerplate
|
|
198
197
|
into the appropriate places. If a new feature file is detected, you will be
|
199
198
|
asked to run `spinach --generate` beforehand.
|
200
199
|
|
201
|
-
**Important**: If auditing individual files, common steps (as above) may be
|
202
|
-
reported as unused when they are actually used in a feature file that is not
|
200
|
+
**Important**: If auditing individual files, common steps (as above) may be
|
201
|
+
reported as unused when they are actually used in a feature file that is not
|
203
202
|
currently being audited. To avoid this, run the audit with no arguments to
|
204
203
|
audit all step files simultaneously.
|
205
204
|
|
206
|
-
|
207
205
|
## Tags
|
208
206
|
|
209
207
|
Feature and Scenarios can be marked with tags in the form: `@tag`. Tags can be
|
@@ -238,19 +236,19 @@ Feature: So something great
|
|
238
236
|
Scenario: Ensure no regression on this
|
239
237
|
```
|
240
238
|
|
241
|
-
|
239
|
+
Then you can run all Scenarios in your suite tagged `@feat-1` using:
|
242
240
|
|
243
241
|
```shell
|
244
242
|
$ spinach --tags @feat-1
|
245
243
|
```
|
246
244
|
|
247
|
-
|
245
|
+
Or only Scenarios tagged either `@feat-1` or `@bug-12` using:
|
248
246
|
|
249
247
|
```shell
|
250
248
|
$ spinach --tags @feat-1,@bug-12
|
251
249
|
```
|
252
250
|
|
253
|
-
|
251
|
+
Or only Scenarios tagged `@feat-1` that aren't tagged `@bug-12` using:
|
254
252
|
|
255
253
|
```shell
|
256
254
|
$ spinach --tags @feat-1,~@bug-12
|
@@ -292,7 +290,7 @@ Full hook documentation is here:
|
|
292
290
|
|
293
291
|
## Local Before and After Hooks
|
294
292
|
|
295
|
-
Sometimes it feels awkward to add steps into feature file just because you need to do some test setup and cleanup. And it is equally awkward to add a global hooks for this purpose. For example, if you want to add a session timeout feature, to do so, you want to set the session timeout time to 1 second just for this feature, and put the normal timeout back after this feature. It doesn't make sense to add two steps in the feature file just to change the session timeout value. In this scenario, a
|
293
|
+
Sometimes it feels awkward to add steps into feature file just because you need to do some test setup and cleanup. And it is equally awkward to add a global hooks for this purpose. For example, if you want to add a session timeout feature, to do so, you want to set the session timeout time to 1 second just for this feature, and put the normal timeout back after this feature. It doesn't make sense to add two steps in the feature file just to change the session timeout value. In this scenario, a `before` and `after` blocks are perfect for this kind of tasks. Below is an example implementation:
|
296
294
|
|
297
295
|
```ruby
|
298
296
|
class Spinach::Features::SessionTimeout < Spinach::FeatureSteps
|
@@ -329,8 +327,8 @@ When no reporter is specified, `stdout` will be used by default.
|
|
329
327
|
|
330
328
|
Other reporters:
|
331
329
|
|
332
|
-
|
333
|
-
|
330
|
+
- For a console reporter with no colors, try [spinach-console-reporter][spinach-console-reporter] (to be used with Jenkins)
|
331
|
+
- For a rerun reporter, try [spinach-rerun-reporter][spinach-rerun-reporter] (writes failed scenarios in a file)
|
334
332
|
|
335
333
|
## Wanna use it with Rails 3?
|
336
334
|
|
@@ -342,46 +340,45 @@ Check out our [spinach-sinatra demo](https://github.com/codegram/spinach-sinatra
|
|
342
340
|
|
343
341
|
## Resources
|
344
342
|
|
345
|
-
|
346
|
-
|
347
|
-
|
348
|
-
|
349
|
-
|
343
|
+
- [Landing page](http://codegram.github.com/spinach)
|
344
|
+
- [Slides](http://codegram.github.com/spinach-presentation)
|
345
|
+
- [Blog post](http://blog.codegram.com/2011/10/how-to-achieve-more-clean-encapsulated-modular-step-definitions-with-spinach)
|
346
|
+
- [API Documentation](http://rubydoc.info/github/codegram/spinach/master/frames)
|
347
|
+
- [Google group](https://groups.google.com/forum/#!forum/spinach_bdd)
|
350
348
|
|
351
349
|
### Related gems
|
352
350
|
|
353
|
-
|
354
|
-
|
355
|
-
|
356
|
-
|
357
|
-
|
351
|
+
- [guard-spinach](http://github.com/codegram/guard-spinach)
|
352
|
+
- [spinach-rails](http://github.com/codegram/spinach-rails)
|
353
|
+
- [spinach-console-reporter][spinach-console-reporter] (to be used with Jenkins)
|
354
|
+
- [spinach-rerun-reporter][spinach-rerun-reporter] (writes failed scenarios in a file)
|
355
|
+
- [spring-commands-spinach](https://github.com/jvanbaarsen/spring-commands-spinach) (to be used with spring)
|
358
356
|
|
359
357
|
### Demos
|
360
358
|
|
361
|
-
|
362
|
-
|
363
|
-
|
359
|
+
- [spinach rails demo](https://github.com/codegram/spinach-rails-demo)
|
360
|
+
- [spinach sinatra demo](https://github.com/codegram/spinach-sinatra-demo)
|
361
|
+
- [simple todo Rails app](https://github.com/codegram/tasca-spinach-demo)
|
364
362
|
|
365
363
|
## Contributing
|
366
364
|
|
367
|
-
|
365
|
+
- [List of spinach contributors](https://github.com/codegram/spinach/contributors)
|
368
366
|
|
369
367
|
You can easily contribute to Spinach. Its codebase is simple and
|
370
368
|
[extensively documented][documentation].
|
371
369
|
|
372
|
-
|
373
|
-
|
374
|
-
|
370
|
+
- Fork the project.
|
371
|
+
- Make your feature addition or bug fix.
|
372
|
+
- Add specs for it. This is important so we don't break it in a future
|
375
373
|
version unintentionally.
|
376
|
-
|
374
|
+
- Commit, do not mess with rakefile, version, or history.
|
377
375
|
If you want to have your own version, that is fine but bump version
|
378
376
|
in a commit by itself I can ignore when I pull.
|
379
|
-
|
377
|
+
- Send me a pull request. Bonus points for topic branches.
|
380
378
|
|
381
379
|
## License
|
382
380
|
|
383
|
-
MIT (Expat) License. Copyright 2011-
|
384
|
-
|
381
|
+
MIT (Expat) License. Copyright 2011-2023 [Codegram Technologies](http://codegram.com)
|
385
382
|
|
386
383
|
[gherkin]: http://github.com/codegram/gherkin-ruby
|
387
384
|
[cucumber]: http://github.com/cucumber/cucumber
|
data/Rakefile
CHANGED
data/bin/spinach
CHANGED
@@ -0,0 +1,22 @@
|
|
1
|
+
Feature: Feature Hooks and Tags
|
2
|
+
In order to run only the appropriate setup and teardown code
|
3
|
+
As a developer
|
4
|
+
I want spinach to only run feature hooks if those features would be run under the tags I provided
|
5
|
+
|
6
|
+
Scenario: No tags specified
|
7
|
+
Given I have a tagged feature with an untagged scenario
|
8
|
+
And I have an untagged feature with a tagged scenario
|
9
|
+
When I don't specify tags
|
10
|
+
Then all the feature hooks should have run
|
11
|
+
|
12
|
+
Scenario: Tags specified
|
13
|
+
Given I have a tagged feature with an untagged scenario
|
14
|
+
And I have an untagged feature with a tagged scenario
|
15
|
+
When I specify a tag the features and scenarios are tagged with
|
16
|
+
Then all the feature hooks should have run
|
17
|
+
|
18
|
+
Scenario: Tags excluded
|
19
|
+
Given I have a tagged feature with an untagged scenario
|
20
|
+
And I have an untagged feature with a tagged scenario
|
21
|
+
When I exclude a tag the features and scenarios are tagged with
|
22
|
+
Then no feature hooks should have run
|
@@ -0,0 +1,16 @@
|
|
1
|
+
Feature: Randomizing Features & Scenarios
|
2
|
+
In order to ensure my tests aren't dependent
|
3
|
+
As a developer
|
4
|
+
I want spinach to randomize features and scenarios (but not steps)
|
5
|
+
|
6
|
+
Scenario: Randomizing the run without specifying a seed
|
7
|
+
Given I have 2 features with 2 scenarios each
|
8
|
+
When I randomize the run without specifying a seed
|
9
|
+
Then The features and scenarios are run
|
10
|
+
And The runner output shows a seed
|
11
|
+
|
12
|
+
Scenario: Specifying the seed
|
13
|
+
Given I have 2 features with 2 scenarios each
|
14
|
+
When I specify the seed for the run
|
15
|
+
Then The features and scenarios are run in a different order
|
16
|
+
And The runner output shows the seed
|
@@ -1,9 +1,25 @@
|
|
1
1
|
Feature: Display run summary
|
2
2
|
As a developer
|
3
3
|
I want spinach to display a summary of steps statuses
|
4
|
-
So I can
|
4
|
+
So I can easily know general features status
|
5
5
|
|
6
|
-
Scenario: Display run summary at the end of features run
|
6
|
+
Scenario: Display run summary at the end of features run without randomization
|
7
7
|
Given I have a feature that has some successful, undefined, failed and error steps
|
8
|
-
|
8
|
+
|
9
|
+
When I run it without randomization
|
10
|
+
Then I should see a summary with steps status information
|
11
|
+
And I shouldn't see a randomization seed
|
12
|
+
|
13
|
+
Scenario: Display run summary at the end of features run with randomization
|
14
|
+
Given I have a feature that has some successful, undefined, failed and error steps
|
15
|
+
|
16
|
+
When I run it with randomization
|
17
|
+
Then I should see a summary with steps status information
|
18
|
+
And I should see a randomization seed
|
19
|
+
|
20
|
+
Scenario: Display run summary at the end of features run with a randomization seed
|
21
|
+
Given I have a feature that has some successful, undefined, failed and error steps
|
22
|
+
|
23
|
+
When I run it with a specific randomization seed
|
9
24
|
Then I should see a summary with steps status information
|
25
|
+
And I should see that specific randomization seed
|
@@ -20,7 +20,7 @@ Feature: Cheezburger can I has
|
|
20
20
|
Then 'I a feature should exist named "features/steps/cheezburger_can_i_has.rb"' do
|
21
21
|
in_current_dir do
|
22
22
|
@file = 'features/steps/cheezburger_can_i_has.rb'
|
23
|
-
File.
|
23
|
+
File.exist?(@file).must_equal true
|
24
24
|
end
|
25
25
|
end
|
26
26
|
|
@@ -0,0 +1,55 @@
|
|
1
|
+
class Spinach::Features::FeatureHooksAndTags < Spinach::FeatureSteps
|
2
|
+
include Integration::SpinachRunner
|
3
|
+
|
4
|
+
step 'I have a tagged feature with an untagged scenario' do
|
5
|
+
write_file 'features/a.feature', <<-FEATURE
|
6
|
+
@tag
|
7
|
+
Feature: A
|
8
|
+
Scenario: A1
|
9
|
+
Then a1
|
10
|
+
FEATURE
|
11
|
+
|
12
|
+
write_file 'features/steps/a.rb', <<-STEPS
|
13
|
+
class Spinach::Features::A < Spinach::FeatureSteps
|
14
|
+
step 'a1' do; end
|
15
|
+
end
|
16
|
+
STEPS
|
17
|
+
end
|
18
|
+
|
19
|
+
step 'I have an untagged feature with a tagged scenario' do
|
20
|
+
write_file 'features/b.feature', <<-FEATURE
|
21
|
+
Feature: B
|
22
|
+
@tag
|
23
|
+
Scenario: B1
|
24
|
+
Then b1
|
25
|
+
FEATURE
|
26
|
+
|
27
|
+
write_file 'features/steps/b.rb', <<-STEPS
|
28
|
+
class Spinach::Features::B < Spinach::FeatureSteps
|
29
|
+
step 'b1' do; end
|
30
|
+
end
|
31
|
+
STEPS
|
32
|
+
end
|
33
|
+
|
34
|
+
step "I don't specify tags" do
|
35
|
+
run_spinach
|
36
|
+
end
|
37
|
+
|
38
|
+
step 'I specify a tag the features and scenarios are tagged with' do
|
39
|
+
run_spinach({append: "--tags @tag"})
|
40
|
+
end
|
41
|
+
|
42
|
+
step 'I exclude a tag the features and scenarios are tagged with' do
|
43
|
+
run_spinach({append: "--tags ~@tag"})
|
44
|
+
end
|
45
|
+
|
46
|
+
step 'all the feature hooks should have run' do
|
47
|
+
@stdout.must_match("Feature: A")
|
48
|
+
@stdout.must_match("Feature: B")
|
49
|
+
end
|
50
|
+
|
51
|
+
step 'no feature hooks should have run' do
|
52
|
+
@stdout.wont_match("Feature: A")
|
53
|
+
@stdout.wont_match("Feature: B")
|
54
|
+
end
|
55
|
+
end
|
@@ -0,0 +1,68 @@
|
|
1
|
+
class Spinach::Features::RandomizingFeaturesScenarios < Spinach::FeatureSteps
|
2
|
+
include Integration::SpinachRunner
|
3
|
+
|
4
|
+
step 'I have 2 features with 2 scenarios each' do
|
5
|
+
write_file 'features/success_a.feature', <<-FEATURE
|
6
|
+
Feature: Success A
|
7
|
+
Scenario: A1
|
8
|
+
Then a1
|
9
|
+
Scenario: A2
|
10
|
+
Then a2
|
11
|
+
FEATURE
|
12
|
+
|
13
|
+
write_file 'features/steps/success_a.rb', <<-STEPS
|
14
|
+
class Spinach::Features::SuccessA < Spinach::FeatureSteps
|
15
|
+
step 'a1' do; end
|
16
|
+
step 'a2' do; end
|
17
|
+
end
|
18
|
+
STEPS
|
19
|
+
|
20
|
+
write_file 'features/success_b.feature', <<-FEATURE
|
21
|
+
Feature: Success B
|
22
|
+
Scenario: B1
|
23
|
+
Then b1
|
24
|
+
Scenario: B2
|
25
|
+
Then b2
|
26
|
+
FEATURE
|
27
|
+
|
28
|
+
write_file 'features/steps/success_b.rb', <<-STEPS
|
29
|
+
class Spinach::Features::SuccessB < Spinach::FeatureSteps
|
30
|
+
step 'b1' do; end
|
31
|
+
step 'b2' do; end
|
32
|
+
end
|
33
|
+
STEPS
|
34
|
+
end
|
35
|
+
|
36
|
+
step 'I randomize the run without specifying a seed' do
|
37
|
+
run_spinach({append: "--rand"})
|
38
|
+
end
|
39
|
+
|
40
|
+
step 'I specify the seed for the run' do
|
41
|
+
# Reverse order (A2 A1 B2 B1) is the only way I can show that
|
42
|
+
# scenarios and features are randomized by the seed when the
|
43
|
+
# example has 2 features each with 2 scenarios. I tried seeds
|
44
|
+
# until I found one that ordered the test in that order.
|
45
|
+
@seed = 1
|
46
|
+
|
47
|
+
run_spinach({append: "--seed #{@seed}"})
|
48
|
+
end
|
49
|
+
|
50
|
+
step 'The features and scenarios are run' do
|
51
|
+
@stdout.must_include("A1")
|
52
|
+
@stdout.must_include("A2")
|
53
|
+
@stdout.must_include("B1")
|
54
|
+
@stdout.must_include("B2")
|
55
|
+
end
|
56
|
+
|
57
|
+
step 'The features and scenarios are run in a different order' do
|
58
|
+
@stdout.must_match(/B2.*B1.*A2.*A1/m)
|
59
|
+
end
|
60
|
+
|
61
|
+
step 'The runner output shows a seed' do
|
62
|
+
@stdout.must_match(/^Randomized with seed \d*$/)
|
63
|
+
end
|
64
|
+
|
65
|
+
step 'The runner output shows the seed' do
|
66
|
+
@stdout.must_match(/^Randomized with seed #{@seed}$/)
|
67
|
+
end
|
68
|
+
end
|
@@ -62,7 +62,7 @@ Feature: A test feature
|
|
62
62
|
@feature = "features/test_feature.feature"
|
63
63
|
end
|
64
64
|
|
65
|
-
When "I run it" do
|
65
|
+
When "I run it without randomization" do
|
66
66
|
run_feature @feature
|
67
67
|
end
|
68
68
|
|
@@ -71,4 +71,32 @@ Feature: A test feature
|
|
71
71
|
/Summary:.*4.*Successful.*1.*Undefined.*1.*Failed.*1.*Error/
|
72
72
|
)
|
73
73
|
end
|
74
|
+
|
75
|
+
And "I shouldn't see a randomization seed" do
|
76
|
+
@stdout.wont_match(
|
77
|
+
/Randomized\ with\ seed\ \d+/
|
78
|
+
)
|
79
|
+
end
|
80
|
+
|
81
|
+
When "I run it with randomization" do
|
82
|
+
run_feature @feature, {append: "--rand"}
|
83
|
+
end
|
84
|
+
|
85
|
+
And "I should see a randomization seed" do
|
86
|
+
@stdout.must_match(
|
87
|
+
/Randomized\ with\ seed\ \d+/
|
88
|
+
)
|
89
|
+
end
|
90
|
+
|
91
|
+
When "I run it with a specific randomization seed" do
|
92
|
+
@seed = rand(0xFFFF)
|
93
|
+
|
94
|
+
run_feature @feature, {append: "--seed #{@seed}"}
|
95
|
+
end
|
96
|
+
|
97
|
+
And "I should see that specific randomization seed" do
|
98
|
+
@stdout.must_match(
|
99
|
+
/Randomized\ with\ seed\ #{@seed}/
|
100
|
+
)
|
101
|
+
end
|
74
102
|
end
|
data/features/support/env.rb
CHANGED
@@ -1,19 +1,6 @@
|
|
1
1
|
require 'minitest/autorun'
|
2
2
|
require 'minitest/spec'
|
3
3
|
require_relative 'filesystem'
|
4
|
-
require 'simplecov'
|
5
|
-
|
6
|
-
if ENV['CI'] && !defined?(Rubinius)
|
7
|
-
require 'coveralls'
|
8
|
-
SimpleCov.formatter = Coveralls::SimpleCov::Formatter
|
9
|
-
require 'simplecov'
|
10
|
-
|
11
|
-
SimpleCov.start do
|
12
|
-
add_filter '/test/'
|
13
|
-
add_filter '/features/'
|
14
|
-
end
|
15
|
-
end
|
16
|
-
|
17
4
|
|
18
5
|
Spinach.hooks.after_scenario do |scenario|
|
19
6
|
FileUtils.rm_rf(Filesystem.dirs)
|
@@ -23,6 +23,17 @@ module Integration
|
|
23
23
|
run "#{ruby} #{spinach} #{feature} #{options[:append]}", options[:env]
|
24
24
|
end
|
25
25
|
|
26
|
+
def run_spinach(options = {})
|
27
|
+
options[:framework] ||= :minitest
|
28
|
+
|
29
|
+
use_minitest if options[:framework] == :minitest
|
30
|
+
use_rspec if options[:framework] == :rspec
|
31
|
+
|
32
|
+
spinach = File.expand_path("bin/spinach")
|
33
|
+
|
34
|
+
run "#{ruby} #{spinach} #{options[:append]}"
|
35
|
+
end
|
36
|
+
|
26
37
|
def ruby
|
27
38
|
return @ruby if defined?(@ruby)
|
28
39
|
|
data/lib/spinach/cli.rb
CHANGED
@@ -56,7 +56,7 @@ module Spinach
|
|
56
56
|
|
57
57
|
@args.each do |arg|
|
58
58
|
if arg.match(/\.feature/)
|
59
|
-
if File.
|
59
|
+
if File.exist? arg.gsub(/:\d*/, '')
|
60
60
|
files_to_run << arg
|
61
61
|
else
|
62
62
|
fail! "#{arg} could not be found"
|
@@ -131,6 +131,16 @@ module Spinach
|
|
131
131
|
config[:reporter_classes] = names
|
132
132
|
end
|
133
133
|
|
134
|
+
opts.on('--rand', "Randomize the order of features and scenarios") do
|
135
|
+
config[:orderer_class] = orderer_class(:random)
|
136
|
+
end
|
137
|
+
|
138
|
+
opts.on('--seed SEED', Integer,
|
139
|
+
"Provide a seed for randomizing the order of features and scenarios") do |seed|
|
140
|
+
config[:orderer_class] = orderer_class(:random)
|
141
|
+
config[:seed] = seed
|
142
|
+
end
|
143
|
+
|
134
144
|
opts.on_tail('--fail-fast',
|
135
145
|
'Terminate the suite run on the first failure') do |class_name|
|
136
146
|
config[:fail_fast] = true
|
@@ -164,7 +174,7 @@ and obsolete steps") do
|
|
164
174
|
# Builds the class name to use an output reporter.
|
165
175
|
#
|
166
176
|
# @param [String] klass
|
167
|
-
# The class name
|
177
|
+
# The class name of the reporter.
|
168
178
|
#
|
169
179
|
# @return [String]
|
170
180
|
# The full name of the reporter class.
|
@@ -177,5 +187,22 @@ and obsolete steps") do
|
|
177
187
|
def reporter_class(klass)
|
178
188
|
"Spinach::Reporter::" + Spinach::Support.camelize(klass)
|
179
189
|
end
|
190
|
+
|
191
|
+
# Builds the class to use an orderer.
|
192
|
+
#
|
193
|
+
# @param [String] klass
|
194
|
+
# The class name of the orderer.
|
195
|
+
#
|
196
|
+
# @return [String]
|
197
|
+
# The full name of the orderer class.
|
198
|
+
#
|
199
|
+
# @example
|
200
|
+
# orderer_class('random')
|
201
|
+
# # => Spinach::Orderers::Random
|
202
|
+
#
|
203
|
+
# @api private
|
204
|
+
def orderer_class(klass)
|
205
|
+
"Spinach::Orderers::" + Spinach::Support.camelize(klass)
|
206
|
+
end
|
180
207
|
end
|
181
208
|
end
|
data/lib/spinach/config.rb
CHANGED
@@ -33,6 +33,8 @@ module Spinach
|
|
33
33
|
:save_and_open_page_on_failure,
|
34
34
|
:reporter_classes,
|
35
35
|
:reporter_options,
|
36
|
+
:orderer_class,
|
37
|
+
:seed,
|
36
38
|
:fail_fast,
|
37
39
|
:audit
|
38
40
|
|
@@ -66,6 +68,29 @@ module Spinach
|
|
66
68
|
@reporter_options || {}
|
67
69
|
end
|
68
70
|
|
71
|
+
# The "orderer class" holds the orderer class name
|
72
|
+
# Defaults to Spinach::Orderers::Default
|
73
|
+
#
|
74
|
+
# @return [orderer object]
|
75
|
+
# The orderer that responds to specific messages.
|
76
|
+
#
|
77
|
+
# @api public
|
78
|
+
def orderer_class
|
79
|
+
@orderer_class || "Spinach::Orderers::Default"
|
80
|
+
end
|
81
|
+
|
82
|
+
# A randomization seed. This is what spinach uses for test run
|
83
|
+
# randomization, so if you call `Kernel.srand(Spinach.config.seed)`
|
84
|
+
# in your support environment file, not only will the test run
|
85
|
+
# order be guaranteed to be stable under a specific seed, all
|
86
|
+
# the Ruby-generated random numbers produced during your test
|
87
|
+
# run will also be stable under that seed.
|
88
|
+
#
|
89
|
+
# @api public
|
90
|
+
def seed
|
91
|
+
@seed ||= rand(0xFFFF)
|
92
|
+
end
|
93
|
+
|
69
94
|
# The "step definitions path" holds the place where your feature step
|
70
95
|
# classes will be searched for. Defaults to '#{features_path}/steps'
|
71
96
|
#
|
data/lib/spinach/feature.rb
CHANGED
@@ -24,6 +24,14 @@ module Spinach
|
|
24
24
|
lines_to_run.empty?
|
25
25
|
end
|
26
26
|
|
27
|
+
# Identifier used by orderers.
|
28
|
+
#
|
29
|
+
# Needs to involve the relative file path so that the ordering
|
30
|
+
# a seed generates is stable across both runs and machines.
|
31
|
+
#
|
32
|
+
# @api public
|
33
|
+
alias ordering_id filename
|
34
|
+
|
27
35
|
# Run the provided code for every step
|
28
36
|
def each_step
|
29
37
|
scenarios.each { |scenario| scenario.steps.each { |step| yield step } }
|
@@ -0,0 +1,25 @@
|
|
1
|
+
module Spinach
|
2
|
+
module Orderers
|
3
|
+
class Default
|
4
|
+
def initialize(**options); end
|
5
|
+
|
6
|
+
# Appends any necessary report output (by default does nothing).
|
7
|
+
#
|
8
|
+
# @param [IO] io
|
9
|
+
# Output buffer for report.
|
10
|
+
#
|
11
|
+
# @api public
|
12
|
+
def attach_summary(io); end
|
13
|
+
|
14
|
+
# Returns a reordered version of the provided array
|
15
|
+
#
|
16
|
+
# @param [Array] items
|
17
|
+
# Items to order
|
18
|
+
#
|
19
|
+
# @api public
|
20
|
+
def order(items)
|
21
|
+
items
|
22
|
+
end
|
23
|
+
end
|
24
|
+
end
|
25
|
+
end
|