cucumber 2.3.3 → 2.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (55) hide show
  1. checksums.yaml +4 -4
  2. data/.github/ISSUE_TEMPLATE.md +48 -0
  3. data/.github/PULL_REQUEST_TEMPLATE.md +39 -0
  4. data/.travis.yml +4 -5
  5. data/Gemfile +3 -1
  6. data/History.md +18 -2
  7. data/LICENSE +1 -1
  8. data/README.md +5 -2
  9. data/cucumber.gemspec +2 -13
  10. data/examples/i18n/eo/features/adicio.feature +6 -6
  11. data/examples/i18n/eo/features/divido.feature +6 -6
  12. data/examples/i18n/eo/features/step_definitions/calculator_steps.rb +2 -2
  13. data/features/docs/cli/randomize.feature +91 -15
  14. data/features/docs/cli/retry_failing_tests.feature +32 -0
  15. data/features/docs/defining_steps/snippets.feature +1 -1
  16. data/features/docs/formatters/json_formatter.feature +2 -6
  17. data/features/docs/formatters/junit_formatter.feature +47 -1
  18. data/features/docs/writing_support_code/after_step_hooks.feature +53 -0
  19. data/features/docs/{defining_steps → writing_support_code}/transforms.feature +42 -7
  20. data/features/lib/step_definitions/cucumber_steps.rb +28 -0
  21. data/features/lib/step_definitions/retry_steps.rb +35 -0
  22. data/lib/cucumber/cli/configuration.rb +4 -0
  23. data/lib/cucumber/cli/options.rb +7 -2
  24. data/lib/cucumber/cli/rerun_file.rb +1 -1
  25. data/lib/cucumber/configuration.rb +4 -0
  26. data/lib/cucumber/events/finished_testing.rb +9 -0
  27. data/lib/cucumber/filters/randomizer.rb +6 -1
  28. data/lib/cucumber/filters/retry.rb +32 -0
  29. data/lib/cucumber/formatter/cucumber.css +0 -0
  30. data/lib/cucumber/formatter/cucumber.sass +0 -0
  31. data/lib/cucumber/formatter/event_bus_report.rb +1 -0
  32. data/lib/cucumber/formatter/json.rb +18 -8
  33. data/lib/cucumber/formatter/junit.rb +56 -42
  34. data/lib/cucumber/gherkin/data_table_parser.rb +17 -9
  35. data/lib/cucumber/gherkin/steps_parser.rb +13 -21
  36. data/lib/cucumber/platform.rb +1 -0
  37. data/lib/cucumber/rake/task.rb +1 -1
  38. data/lib/cucumber/rb_support/rb_hook.rb +1 -1
  39. data/lib/cucumber/rb_support/snippet.rb +1 -1
  40. data/lib/cucumber/runtime.rb +4 -5
  41. data/lib/cucumber/runtime/for_programming_languages.rb +2 -0
  42. data/lib/cucumber/runtime/step_hooks.rb +1 -1
  43. data/lib/cucumber/step_match.rb +1 -1
  44. data/lib/cucumber/version +1 -1
  45. data/spec/cucumber/cli/configuration_spec.rb +7 -0
  46. data/spec/cucumber/cli/options_spec.rb +14 -0
  47. data/spec/cucumber/cli/rerun_spec.rb +3 -7
  48. data/spec/cucumber/filters/retry_spec.rb +79 -0
  49. data/spec/cucumber/formatter/event_bus_report_spec.rb +9 -0
  50. data/spec/cucumber/formatter/json_spec.rb +1 -1
  51. data/spec/cucumber/formatter/junit_spec.rb +2 -2
  52. data/spec/cucumber/formatter/spec_helper.rb +6 -1
  53. data/spec/cucumber/rake/task_spec.rb +85 -0
  54. data/spec/cucumber/rb_support/snippet_spec.rb +2 -2
  55. metadata +19 -139
@@ -50,7 +50,7 @@ Feature: Snippets
50
50
  Then the output should contain:
51
51
  """
52
52
  Given(/^a table$/) do |table|
53
- # table is a Cucumber::Core::Ast::DataTable
53
+ # table is a Cucumber::MultilineArgument::DataTable
54
54
  pending # Write code here that turns the phrase above into concrete actions
55
55
  end
56
56
  """
@@ -286,7 +286,6 @@ Feature: JSON output formatter
286
286
 
287
287
  """
288
288
 
289
- @spawn
290
289
  Scenario: DocString
291
290
  Given a file named "features/doc_string.feature" with:
292
291
  """
@@ -349,9 +348,8 @@ Feature: JSON output formatter
349
348
  ]
350
349
  """
351
350
 
352
- @spawn
353
351
  Scenario: embedding screenshot
354
- When I run `cucumber -b --format json features/embed.feature`
352
+ When I run `cucumber --format json features/embed.feature`
355
353
  Then it should pass with JSON:
356
354
  """
357
355
  [
@@ -630,9 +628,8 @@ Feature: JSON output formatter
630
628
 
631
629
  """
632
630
 
633
- @spawn
634
631
  Scenario: embedding data directly
635
- When I run `cucumber -b --format json -x features/embed_data_directly.feature`
632
+ When I run `cucumber --format json -x features/embed_data_directly.feature`
636
633
  Then it should pass with JSON:
637
634
  """
638
635
  [
@@ -733,7 +730,6 @@ Feature: JSON output formatter
733
730
  ]
734
731
 
735
732
  """
736
- @spawn
737
733
  Scenario: handle output from hooks
738
734
  Given a file named "features/step_definitions/output_steps.rb" with:
739
735
  """
@@ -1,4 +1,3 @@
1
- @spawn
2
1
  Feature: JUnit output formatter
3
2
  In order for developers to create test reports with ant
4
3
  Cucumber should be able to output JUnit xml files
@@ -60,6 +59,7 @@ Feature: JUnit output formatter
60
59
  | is undefined |
61
60
  """
62
61
 
62
+ @spawn
63
63
  Scenario: one feature, one passing scenario, one failing scenario
64
64
  When I run `cucumber --format junit --out tmp/ features/one_passing_one_failing.feature`
65
65
  Then it should fail with:
@@ -101,6 +101,7 @@ Feature: JUnit output formatter
101
101
 
102
102
  """
103
103
 
104
+ @spawn
104
105
  Scenario: one feature in a subdirectory, one passing scenario, one failing scenario
105
106
  When I run `cucumber --format junit --out tmp/ features/some_subdirectory/one_passing_one_failing.feature --require features`
106
107
  Then it should fail with:
@@ -245,6 +246,7 @@ can't convert .* into String \(TypeError\)
245
246
  You *must* specify --out DIR for the junit formatter
246
247
  """
247
248
 
249
+ @spawn
248
250
  Scenario: strict mode, one feature, one scenario outline, four examples: one passing, one failing, one pending, one undefined
249
251
  When I run `cucumber --strict --format junit --out tmp/ features/scenario_outline.feature`
250
252
  Then it should fail with:
@@ -326,6 +328,7 @@ You *must* specify --out DIR for the junit formatter
326
328
 
327
329
  """
328
330
 
331
+ @spawn
329
332
  Scenario: strict mode with --expand option, one feature, one scenario outline, four examples: one passing, one failing, one pending, one undefined
330
333
  When I run `cucumber --strict --expand --format junit --out tmp/ features/scenario_outline.feature`
331
334
  Then it should fail with exactly:
@@ -406,3 +409,46 @@ You *must* specify --out DIR for the junit formatter
406
409
  </testsuite>
407
410
 
408
411
  """
412
+
413
+ @spawn
414
+ Scenario: run test cases from different features interweaved
415
+ When I run `cucumber --format junit --out tmp/ features/one_passing_one_failing.feature:3 features/pending.feature:3 features/one_passing_one_failing.feature:6`
416
+ Then it should fail with:
417
+ """
418
+
419
+ """
420
+ And the junit output file "tmp/TEST-features-one_passing_one_failing.xml" should contain:
421
+ """
422
+ <?xml version="1.0" encoding="UTF-8"?>
423
+ <testsuite failures="1" errors="0" skipped="0" tests="2" time="0.05" name="One passing scenario, one failing scenario">
424
+ <testcase classname="One passing scenario, one failing scenario" name="Passing" time="0.05">
425
+ <system-out>
426
+ <![CDATA[]]>
427
+ </system-out>
428
+ <system-err>
429
+ <![CDATA[]]>
430
+ </system-err>
431
+ </testcase>
432
+ <testcase classname="One passing scenario, one failing scenario" name="Failing" time="0.05">
433
+ <failure message="failed Failing" type="failed">
434
+ <![CDATA[Scenario: Failing
435
+
436
+ Given this step fails
437
+
438
+ Message:
439
+ ]]>
440
+ <![CDATA[ (RuntimeError)
441
+ ./features/step_definitions/steps.rb:4:in `/^this step fails$/'
442
+ features/one_passing_one_failing.feature:7:in `Given this step fails']]>
443
+ </failure>
444
+ <system-out>
445
+ <![CDATA[]]>
446
+ </system-out>
447
+ <system-err>
448
+ <![CDATA[]]>
449
+ </system-err>
450
+ </testcase>
451
+ </testsuite>
452
+
453
+ """
454
+ And a file named "tmp/TEST-features-pending.xml" should exist
@@ -0,0 +1,53 @@
1
+ Feature: AfterStep Hooks
2
+ AfterStep hooks can be used to further inspect the Step object of the step
3
+ that has just run, or to simply check the step's result.
4
+
5
+ Background:
6
+ Given the standard step definitions
7
+ And a file named "features/sample.feature" with:
8
+ """
9
+ Feature: Sample
10
+
11
+ Scenario: Success
12
+ Given this step passes
13
+ """
14
+
15
+ Scenario: Access Test Step object in AfterStep Block
16
+ Given a file named "features/support/env.rb" with:
17
+ """
18
+ AfterStep do |result, test_step|
19
+ expect(test_step).to be_a(Cucumber::Core::Test::Step)
20
+ end
21
+ """
22
+ When I run `cucumber features`
23
+ Then it should pass with:
24
+ """
25
+ Feature: Sample
26
+
27
+ Scenario: Success # features/sample.feature:3
28
+ Given this step passes # features/step_definitions/steps.rb:1
29
+
30
+ 1 scenario (1 passed)
31
+ 1 step (1 passed)
32
+
33
+ """
34
+
35
+ Scenario: An AfterStep with one named argument receives only the result
36
+ Given a file named "features/support/env.rb" with:
37
+ """
38
+ AfterStep do |result|
39
+ expect(result).to be_a(Cucumber::Core::Test::Result::Passed)
40
+ end
41
+ """
42
+ When I run `cucumber features`
43
+ Then it should pass with:
44
+ """
45
+ Feature: Sample
46
+
47
+ Scenario: Success # features/sample.feature:3
48
+ Given this step passes # features/step_definitions/steps.rb:1
49
+
50
+ 1 scenario (1 passed)
51
+ 1 step (1 passed)
52
+
53
+ """
@@ -1,12 +1,12 @@
1
1
  Feature: Transforms
2
2
 
3
- If you see certain phrases repeated over and over in your step definitions, you can
4
- use transforms to factor out that duplication, and make your step definitions simpler.
5
-
3
+ Transforms allow you to convert primitive string arguments captured in step definitions
4
+ into more meaningful data types.
5
+
6
6
  Background:
7
7
  Let's just create a simple feature for testing out Transforms.
8
8
  We also have a Person class that we need to be able to build.
9
-
9
+
10
10
  Given a file named "features/foo.feature" with:
11
11
  """
12
12
  Feature:
@@ -26,8 +26,9 @@ Feature: Transforms
26
26
 
27
27
  Scenario: Basic Transform
28
28
  This is the most basic way to use a transform. Notice that the regular
29
- expression is pretty much duplicated.
30
-
29
+ expression is pretty much duplicated between the step defintion and the
30
+ transform itself.
31
+
31
32
  And a file named "features/step_definitions/steps.rb" with:
32
33
  """
33
34
  Transform(/a Person aged (\d+)/) do |age|
@@ -42,7 +43,7 @@ Feature: Transforms
42
43
  """
43
44
  When I run `cucumber features/foo.feature`
44
45
  Then it should pass
45
-
46
+
46
47
  Scenario: Re-use Transform's Regular Expression
47
48
  If you keep a reference to the transform, you can use it in your
48
49
  regular expressions to avoid repeating the regular expression.
@@ -61,3 +62,37 @@ Feature: Transforms
61
62
  """
62
63
  When I run `cucumber features/foo.feature`
63
64
  Then it should pass
65
+
66
+ Scenario: Use Transform from code
67
+ You can also call the `Transform` method from within a step definition to
68
+ dynamically run a tranform.
69
+
70
+ Note that this is rather useless at present, unless the transform itself has a side-effect, since the `Transform`
71
+ function still returns the regular expression pattern.
72
+
73
+ Given a file named "features/people.feature" with:
74
+ """
75
+ Feature:
76
+ Scenario:
77
+ Given people with these ages:
78
+ | name | age |
79
+ | sue | 25 |
80
+ | sally | 77 |
81
+ """
82
+ And a file named "features/step_definitions/steps.rb" with:
83
+ """
84
+ Transform(/a Person aged (\d+)/) do |age|
85
+ $person = Person.new
86
+ $person.age = age.to_i
87
+ $person
88
+ end
89
+
90
+ Given(/^people with these ages:$/) do |table|
91
+ table.rows.each do |name, age|
92
+ Transform("a Person aged #{age}")
93
+ expect($person.age).to eq age.to_i
94
+ end
95
+ end
96
+ """
97
+ When I run `cucumber features/people.feature`
98
+ Then it should pass
@@ -32,6 +32,34 @@ Given /^a step definition that looks like this:$/ do |string|
32
32
  create_step_definition { string }
33
33
  end
34
34
 
35
+ Given /^a scenario "([^\"]*)" that passes$/ do |name|
36
+ write_file "features/#{name}.feature",
37
+ <<-FEATURE
38
+ Feature: #{name}
39
+ Scenario: #{name}
40
+ Given it passes
41
+ FEATURE
42
+
43
+ write_file "features/step_definitions/#{name}_steps.rb",
44
+ <<-STEPS
45
+ Given(/^it passes$/) { expect(true).to be true }
46
+ STEPS
47
+ end
48
+
49
+ Given /^a scenario "([^\"]*)" that fails$/ do |name|
50
+ write_file "features/#{name}.feature",
51
+ <<-FEATURE
52
+ Feature: #{name}
53
+ Scenario: #{name}
54
+ Given it fails
55
+ FEATURE
56
+
57
+ write_file "features/step_definitions/#{name}_steps.rb",
58
+ <<-STEPS
59
+ Given(/^it fails$/) { expect(false).to be true }
60
+ STEPS
61
+ end
62
+
35
63
  When /^I run the feature with the (\w+) formatter$/ do |formatter|
36
64
  expect(features.length).to eq 1
37
65
  run_feature features.first, formatter
@@ -0,0 +1,35 @@
1
+ Given /^a scenario "([^\"]*)" that fails once, then passes$/ do |name|
2
+ write_file "features/#{name}.feature",
3
+ <<-FEATURE
4
+ Feature: #{name}
5
+ Scenario: #{name}
6
+ Given it fails once, then passes
7
+ FEATURE
8
+
9
+ write_file "features/step_defnitions/#{name}_steps.rb",
10
+ <<-STEPS
11
+ Given(/^it fails once, then passes$/) do
12
+ $#{name.downcase} ||= 0
13
+ $#{name.downcase} += 1
14
+ expect($#{name.downcase}).to eql 2
15
+ end
16
+ STEPS
17
+ end
18
+
19
+ Given /^a scenario "([^\"]*)" that fails twice, then passes$/ do |name|
20
+ write_file "features/#{name}.feature",
21
+ <<-FEATURE
22
+ Feature: #{name}
23
+ Scenario: #{name}
24
+ Given it fails twice, then passes
25
+ FEATURE
26
+
27
+ write_file "features/step_definitions/#{name}_steps.rb",
28
+ <<-STEPS
29
+ Given(/^it fails twice, then passes$/) do
30
+ $#{name.downcase} ||= 0
31
+ $#{name.downcase} += 1
32
+ expect($#{name.downcase}).to eql 3
33
+ end
34
+ STEPS
35
+ end
@@ -67,6 +67,10 @@ module Cucumber
67
67
  !!@options[:fail_fast]
68
68
  end
69
69
 
70
+ def retry_attempts
71
+ @options[:retry]
72
+ end
73
+
70
74
  def snippet_type
71
75
  @options[:snippet_type] || :regexp
72
76
  end
@@ -44,9 +44,10 @@ module Cucumber
44
44
  PROFILE_LONG_FLAG = '--profile'
45
45
  NO_PROFILE_LONG_FLAG = '--no-profile'
46
46
  FAIL_FAST_FLAG = '--fail-fast'
47
+ RETRY_FLAG = '--retry'
47
48
  OPTIONS_WITH_ARGS = ['-r', '--require', '--i18n', '-f', '--format', '-o', '--out',
48
49
  '-t', '--tags', '-n', '--name', '-e', '--exclude',
49
- PROFILE_SHORT_FLAG, PROFILE_LONG_FLAG,
50
+ PROFILE_SHORT_FLAG, PROFILE_LONG_FLAG, RETRY_FLAG,
50
51
  '-l', '--lines', '--port',
51
52
  '-I', '--snippet-type']
52
53
  ORDER_TYPES = %w{defined random}
@@ -188,6 +189,9 @@ module Cucumber
188
189
  "Disables all profile loading to avoid using the 'default' profile.") do |v|
189
190
  @disable_profile_loading = true
190
191
  end
192
+ opts.on("#{RETRY_FLAG} ATTEMPTS", "Specify the number of times to retry failing tests (default: 0)") do |v|
193
+ @options[:retry] = v.to_i
194
+ end
191
195
  opts.on("-c", "--[no-]color",
192
196
  "Whether or not to use ANSI color in the output. Cucumber decides",
193
197
  "based on your platform and the output destination if not specified.") do |v|
@@ -450,7 +454,8 @@ TEXT
450
454
  :diff_enabled => true,
451
455
  :snippets => true,
452
456
  :source => true,
453
- :duration => true
457
+ :duration => true,
458
+ :retry => 0
454
459
  }
455
460
  end
456
461
  end
@@ -16,7 +16,7 @@ module Cucumber
16
16
  end
17
17
 
18
18
  def features
19
- lines.map { |l| l.scan(/(?:^| )(.*?\.feature(?:(?::\d+)*))/) }.flatten
19
+ lines.map { |l| l.scan(/(?:^| |)(.*?\.feature(?:(?::\d+)*))/) }.flatten
20
20
  end
21
21
 
22
22
  private
@@ -65,6 +65,10 @@ module Cucumber
65
65
  @options[:fail_fast]
66
66
  end
67
67
 
68
+ def retry_attempts
69
+ @options[:retry]
70
+ end
71
+
68
72
  def guess?
69
73
  @options[:guess]
70
74
  end
@@ -0,0 +1,9 @@
1
+ module Cucumber
2
+ module Events
3
+
4
+ # Event fired after aall test cases have finished executing
5
+ class FinishedTesting
6
+ end
7
+
8
+ end
9
+ end
@@ -1,3 +1,5 @@
1
+ require 'digest/sha2'
2
+
1
3
  module Cucumber
2
4
  module Filters
3
5
 
@@ -29,7 +31,10 @@ module Cucumber
29
31
  private
30
32
 
31
33
  def shuffled_test_cases
32
- @test_cases.shuffle(random: Random.new(seed))
34
+ digester = Digest::SHA2.new(256)
35
+ @test_cases.map.with_index.
36
+ sort_by { |_, index| digester.digest((@seed + index).to_s) }.
37
+ map { |test_case, _| test_case }
33
38
  end
34
39
 
35
40
  attr_reader :seed
@@ -0,0 +1,32 @@
1
+ require 'cucumber/core/filter'
2
+ require 'cucumber/running_test_case'
3
+ require 'cucumber/events/bus'
4
+ require 'cucumber/events/after_test_case'
5
+
6
+ module Cucumber
7
+ module Filters
8
+ class Retry < Core::Filter.new(:configuration)
9
+
10
+ def test_case(test_case)
11
+ configuration.on_event(:after_test_case) do |event|
12
+ next unless retry_required?(test_case, event)
13
+
14
+ test_case_counts[test_case] += 1
15
+ event.test_case.describe_to(receiver)
16
+ end
17
+
18
+ super
19
+ end
20
+
21
+ private
22
+
23
+ def retry_required?(test_case, event)
24
+ event.test_case == test_case && event.result.failed? && test_case_counts[test_case] < configuration.retry_attempts
25
+ end
26
+
27
+ def test_case_counts
28
+ @test_case_counts ||= Hash.new {|h,k| h[k] = 0 }
29
+ end
30
+ end
31
+ end
32
+ end