cucumber 2.3.3 → 2.4.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (55) hide show
  1. checksums.yaml +4 -4
  2. data/.github/ISSUE_TEMPLATE.md +48 -0
  3. data/.github/PULL_REQUEST_TEMPLATE.md +39 -0
  4. data/.travis.yml +4 -5
  5. data/Gemfile +3 -1
  6. data/History.md +18 -2
  7. data/LICENSE +1 -1
  8. data/README.md +5 -2
  9. data/cucumber.gemspec +2 -13
  10. data/examples/i18n/eo/features/adicio.feature +6 -6
  11. data/examples/i18n/eo/features/divido.feature +6 -6
  12. data/examples/i18n/eo/features/step_definitions/calculator_steps.rb +2 -2
  13. data/features/docs/cli/randomize.feature +91 -15
  14. data/features/docs/cli/retry_failing_tests.feature +32 -0
  15. data/features/docs/defining_steps/snippets.feature +1 -1
  16. data/features/docs/formatters/json_formatter.feature +2 -6
  17. data/features/docs/formatters/junit_formatter.feature +47 -1
  18. data/features/docs/writing_support_code/after_step_hooks.feature +53 -0
  19. data/features/docs/{defining_steps → writing_support_code}/transforms.feature +42 -7
  20. data/features/lib/step_definitions/cucumber_steps.rb +28 -0
  21. data/features/lib/step_definitions/retry_steps.rb +35 -0
  22. data/lib/cucumber/cli/configuration.rb +4 -0
  23. data/lib/cucumber/cli/options.rb +7 -2
  24. data/lib/cucumber/cli/rerun_file.rb +1 -1
  25. data/lib/cucumber/configuration.rb +4 -0
  26. data/lib/cucumber/events/finished_testing.rb +9 -0
  27. data/lib/cucumber/filters/randomizer.rb +6 -1
  28. data/lib/cucumber/filters/retry.rb +32 -0
  29. data/lib/cucumber/formatter/cucumber.css +0 -0
  30. data/lib/cucumber/formatter/cucumber.sass +0 -0
  31. data/lib/cucumber/formatter/event_bus_report.rb +1 -0
  32. data/lib/cucumber/formatter/json.rb +18 -8
  33. data/lib/cucumber/formatter/junit.rb +56 -42
  34. data/lib/cucumber/gherkin/data_table_parser.rb +17 -9
  35. data/lib/cucumber/gherkin/steps_parser.rb +13 -21
  36. data/lib/cucumber/platform.rb +1 -0
  37. data/lib/cucumber/rake/task.rb +1 -1
  38. data/lib/cucumber/rb_support/rb_hook.rb +1 -1
  39. data/lib/cucumber/rb_support/snippet.rb +1 -1
  40. data/lib/cucumber/runtime.rb +4 -5
  41. data/lib/cucumber/runtime/for_programming_languages.rb +2 -0
  42. data/lib/cucumber/runtime/step_hooks.rb +1 -1
  43. data/lib/cucumber/step_match.rb +1 -1
  44. data/lib/cucumber/version +1 -1
  45. data/spec/cucumber/cli/configuration_spec.rb +7 -0
  46. data/spec/cucumber/cli/options_spec.rb +14 -0
  47. data/spec/cucumber/cli/rerun_spec.rb +3 -7
  48. data/spec/cucumber/filters/retry_spec.rb +79 -0
  49. data/spec/cucumber/formatter/event_bus_report_spec.rb +9 -0
  50. data/spec/cucumber/formatter/json_spec.rb +1 -1
  51. data/spec/cucumber/formatter/junit_spec.rb +2 -2
  52. data/spec/cucumber/formatter/spec_helper.rb +6 -1
  53. data/spec/cucumber/rake/task_spec.rb +85 -0
  54. data/spec/cucumber/rb_support/snippet_spec.rb +2 -2
  55. metadata +19 -139
@@ -50,7 +50,7 @@ Feature: Snippets
50
50
  Then the output should contain:
51
51
  """
52
52
  Given(/^a table$/) do |table|
53
- # table is a Cucumber::Core::Ast::DataTable
53
+ # table is a Cucumber::MultilineArgument::DataTable
54
54
  pending # Write code here that turns the phrase above into concrete actions
55
55
  end
56
56
  """
@@ -286,7 +286,6 @@ Feature: JSON output formatter
286
286
 
287
287
  """
288
288
 
289
- @spawn
290
289
  Scenario: DocString
291
290
  Given a file named "features/doc_string.feature" with:
292
291
  """
@@ -349,9 +348,8 @@ Feature: JSON output formatter
349
348
  ]
350
349
  """
351
350
 
352
- @spawn
353
351
  Scenario: embedding screenshot
354
- When I run `cucumber -b --format json features/embed.feature`
352
+ When I run `cucumber --format json features/embed.feature`
355
353
  Then it should pass with JSON:
356
354
  """
357
355
  [
@@ -630,9 +628,8 @@ Feature: JSON output formatter
630
628
 
631
629
  """
632
630
 
633
- @spawn
634
631
  Scenario: embedding data directly
635
- When I run `cucumber -b --format json -x features/embed_data_directly.feature`
632
+ When I run `cucumber --format json -x features/embed_data_directly.feature`
636
633
  Then it should pass with JSON:
637
634
  """
638
635
  [
@@ -733,7 +730,6 @@ Feature: JSON output formatter
733
730
  ]
734
731
 
735
732
  """
736
- @spawn
737
733
  Scenario: handle output from hooks
738
734
  Given a file named "features/step_definitions/output_steps.rb" with:
739
735
  """
@@ -1,4 +1,3 @@
1
- @spawn
2
1
  Feature: JUnit output formatter
3
2
  In order for developers to create test reports with ant
4
3
  Cucumber should be able to output JUnit xml files
@@ -60,6 +59,7 @@ Feature: JUnit output formatter
60
59
  | is undefined |
61
60
  """
62
61
 
62
+ @spawn
63
63
  Scenario: one feature, one passing scenario, one failing scenario
64
64
  When I run `cucumber --format junit --out tmp/ features/one_passing_one_failing.feature`
65
65
  Then it should fail with:
@@ -101,6 +101,7 @@ Feature: JUnit output formatter
101
101
 
102
102
  """
103
103
 
104
+ @spawn
104
105
  Scenario: one feature in a subdirectory, one passing scenario, one failing scenario
105
106
  When I run `cucumber --format junit --out tmp/ features/some_subdirectory/one_passing_one_failing.feature --require features`
106
107
  Then it should fail with:
@@ -245,6 +246,7 @@ can't convert .* into String \(TypeError\)
245
246
  You *must* specify --out DIR for the junit formatter
246
247
  """
247
248
 
249
+ @spawn
248
250
  Scenario: strict mode, one feature, one scenario outline, four examples: one passing, one failing, one pending, one undefined
249
251
  When I run `cucumber --strict --format junit --out tmp/ features/scenario_outline.feature`
250
252
  Then it should fail with:
@@ -326,6 +328,7 @@ You *must* specify --out DIR for the junit formatter
326
328
 
327
329
  """
328
330
 
331
+ @spawn
329
332
  Scenario: strict mode with --expand option, one feature, one scenario outline, four examples: one passing, one failing, one pending, one undefined
330
333
  When I run `cucumber --strict --expand --format junit --out tmp/ features/scenario_outline.feature`
331
334
  Then it should fail with exactly:
@@ -406,3 +409,46 @@ You *must* specify --out DIR for the junit formatter
406
409
  </testsuite>
407
410
 
408
411
  """
412
+
413
+ @spawn
414
+ Scenario: run test cases from different features interweaved
415
+ When I run `cucumber --format junit --out tmp/ features/one_passing_one_failing.feature:3 features/pending.feature:3 features/one_passing_one_failing.feature:6`
416
+ Then it should fail with:
417
+ """
418
+
419
+ """
420
+ And the junit output file "tmp/TEST-features-one_passing_one_failing.xml" should contain:
421
+ """
422
+ <?xml version="1.0" encoding="UTF-8"?>
423
+ <testsuite failures="1" errors="0" skipped="0" tests="2" time="0.05" name="One passing scenario, one failing scenario">
424
+ <testcase classname="One passing scenario, one failing scenario" name="Passing" time="0.05">
425
+ <system-out>
426
+ <![CDATA[]]>
427
+ </system-out>
428
+ <system-err>
429
+ <![CDATA[]]>
430
+ </system-err>
431
+ </testcase>
432
+ <testcase classname="One passing scenario, one failing scenario" name="Failing" time="0.05">
433
+ <failure message="failed Failing" type="failed">
434
+ <![CDATA[Scenario: Failing
435
+
436
+ Given this step fails
437
+
438
+ Message:
439
+ ]]>
440
+ <![CDATA[ (RuntimeError)
441
+ ./features/step_definitions/steps.rb:4:in `/^this step fails$/'
442
+ features/one_passing_one_failing.feature:7:in `Given this step fails']]>
443
+ </failure>
444
+ <system-out>
445
+ <![CDATA[]]>
446
+ </system-out>
447
+ <system-err>
448
+ <![CDATA[]]>
449
+ </system-err>
450
+ </testcase>
451
+ </testsuite>
452
+
453
+ """
454
+ And a file named "tmp/TEST-features-pending.xml" should exist
@@ -0,0 +1,53 @@
1
+ Feature: AfterStep Hooks
2
+ AfterStep hooks can be used to further inspect the Step object of the step
3
+ that has just run, or to simply check the step's result.
4
+
5
+ Background:
6
+ Given the standard step definitions
7
+ And a file named "features/sample.feature" with:
8
+ """
9
+ Feature: Sample
10
+
11
+ Scenario: Success
12
+ Given this step passes
13
+ """
14
+
15
+ Scenario: Access Test Step object in AfterStep Block
16
+ Given a file named "features/support/env.rb" with:
17
+ """
18
+ AfterStep do |result, test_step|
19
+ expect(test_step).to be_a(Cucumber::Core::Test::Step)
20
+ end
21
+ """
22
+ When I run `cucumber features`
23
+ Then it should pass with:
24
+ """
25
+ Feature: Sample
26
+
27
+ Scenario: Success # features/sample.feature:3
28
+ Given this step passes # features/step_definitions/steps.rb:1
29
+
30
+ 1 scenario (1 passed)
31
+ 1 step (1 passed)
32
+
33
+ """
34
+
35
+ Scenario: An AfterStep with one named argument receives only the result
36
+ Given a file named "features/support/env.rb" with:
37
+ """
38
+ AfterStep do |result|
39
+ expect(result).to be_a(Cucumber::Core::Test::Result::Passed)
40
+ end
41
+ """
42
+ When I run `cucumber features`
43
+ Then it should pass with:
44
+ """
45
+ Feature: Sample
46
+
47
+ Scenario: Success # features/sample.feature:3
48
+ Given this step passes # features/step_definitions/steps.rb:1
49
+
50
+ 1 scenario (1 passed)
51
+ 1 step (1 passed)
52
+
53
+ """
@@ -1,12 +1,12 @@
1
1
  Feature: Transforms
2
2
 
3
- If you see certain phrases repeated over and over in your step definitions, you can
4
- use transforms to factor out that duplication, and make your step definitions simpler.
5
-
3
+ Transforms allow you to convert primitive string arguments captured in step definitions
4
+ into more meaningful data types.
5
+
6
6
  Background:
7
7
  Let's just create a simple feature for testing out Transforms.
8
8
  We also have a Person class that we need to be able to build.
9
-
9
+
10
10
  Given a file named "features/foo.feature" with:
11
11
  """
12
12
  Feature:
@@ -26,8 +26,9 @@ Feature: Transforms
26
26
 
27
27
  Scenario: Basic Transform
28
28
  This is the most basic way to use a transform. Notice that the regular
29
- expression is pretty much duplicated.
30
-
29
+ expression is pretty much duplicated between the step defintion and the
30
+ transform itself.
31
+
31
32
  And a file named "features/step_definitions/steps.rb" with:
32
33
  """
33
34
  Transform(/a Person aged (\d+)/) do |age|
@@ -42,7 +43,7 @@ Feature: Transforms
42
43
  """
43
44
  When I run `cucumber features/foo.feature`
44
45
  Then it should pass
45
-
46
+
46
47
  Scenario: Re-use Transform's Regular Expression
47
48
  If you keep a reference to the transform, you can use it in your
48
49
  regular expressions to avoid repeating the regular expression.
@@ -61,3 +62,37 @@ Feature: Transforms
61
62
  """
62
63
  When I run `cucumber features/foo.feature`
63
64
  Then it should pass
65
+
66
+ Scenario: Use Transform from code
67
+ You can also call the `Transform` method from within a step definition to
68
+ dynamically run a tranform.
69
+
70
+ Note that this is rather useless at present, unless the transform itself has a side-effect, since the `Transform`
71
+ function still returns the regular expression pattern.
72
+
73
+ Given a file named "features/people.feature" with:
74
+ """
75
+ Feature:
76
+ Scenario:
77
+ Given people with these ages:
78
+ | name | age |
79
+ | sue | 25 |
80
+ | sally | 77 |
81
+ """
82
+ And a file named "features/step_definitions/steps.rb" with:
83
+ """
84
+ Transform(/a Person aged (\d+)/) do |age|
85
+ $person = Person.new
86
+ $person.age = age.to_i
87
+ $person
88
+ end
89
+
90
+ Given(/^people with these ages:$/) do |table|
91
+ table.rows.each do |name, age|
92
+ Transform("a Person aged #{age}")
93
+ expect($person.age).to eq age.to_i
94
+ end
95
+ end
96
+ """
97
+ When I run `cucumber features/people.feature`
98
+ Then it should pass
@@ -32,6 +32,34 @@ Given /^a step definition that looks like this:$/ do |string|
32
32
  create_step_definition { string }
33
33
  end
34
34
 
35
+ Given /^a scenario "([^\"]*)" that passes$/ do |name|
36
+ write_file "features/#{name}.feature",
37
+ <<-FEATURE
38
+ Feature: #{name}
39
+ Scenario: #{name}
40
+ Given it passes
41
+ FEATURE
42
+
43
+ write_file "features/step_definitions/#{name}_steps.rb",
44
+ <<-STEPS
45
+ Given(/^it passes$/) { expect(true).to be true }
46
+ STEPS
47
+ end
48
+
49
+ Given /^a scenario "([^\"]*)" that fails$/ do |name|
50
+ write_file "features/#{name}.feature",
51
+ <<-FEATURE
52
+ Feature: #{name}
53
+ Scenario: #{name}
54
+ Given it fails
55
+ FEATURE
56
+
57
+ write_file "features/step_definitions/#{name}_steps.rb",
58
+ <<-STEPS
59
+ Given(/^it fails$/) { expect(false).to be true }
60
+ STEPS
61
+ end
62
+
35
63
  When /^I run the feature with the (\w+) formatter$/ do |formatter|
36
64
  expect(features.length).to eq 1
37
65
  run_feature features.first, formatter
@@ -0,0 +1,35 @@
1
+ Given /^a scenario "([^\"]*)" that fails once, then passes$/ do |name|
2
+ write_file "features/#{name}.feature",
3
+ <<-FEATURE
4
+ Feature: #{name}
5
+ Scenario: #{name}
6
+ Given it fails once, then passes
7
+ FEATURE
8
+
9
+ write_file "features/step_defnitions/#{name}_steps.rb",
10
+ <<-STEPS
11
+ Given(/^it fails once, then passes$/) do
12
+ $#{name.downcase} ||= 0
13
+ $#{name.downcase} += 1
14
+ expect($#{name.downcase}).to eql 2
15
+ end
16
+ STEPS
17
+ end
18
+
19
+ Given /^a scenario "([^\"]*)" that fails twice, then passes$/ do |name|
20
+ write_file "features/#{name}.feature",
21
+ <<-FEATURE
22
+ Feature: #{name}
23
+ Scenario: #{name}
24
+ Given it fails twice, then passes
25
+ FEATURE
26
+
27
+ write_file "features/step_definitions/#{name}_steps.rb",
28
+ <<-STEPS
29
+ Given(/^it fails twice, then passes$/) do
30
+ $#{name.downcase} ||= 0
31
+ $#{name.downcase} += 1
32
+ expect($#{name.downcase}).to eql 3
33
+ end
34
+ STEPS
35
+ end
@@ -67,6 +67,10 @@ module Cucumber
67
67
  !!@options[:fail_fast]
68
68
  end
69
69
 
70
+ def retry_attempts
71
+ @options[:retry]
72
+ end
73
+
70
74
  def snippet_type
71
75
  @options[:snippet_type] || :regexp
72
76
  end
@@ -44,9 +44,10 @@ module Cucumber
44
44
  PROFILE_LONG_FLAG = '--profile'
45
45
  NO_PROFILE_LONG_FLAG = '--no-profile'
46
46
  FAIL_FAST_FLAG = '--fail-fast'
47
+ RETRY_FLAG = '--retry'
47
48
  OPTIONS_WITH_ARGS = ['-r', '--require', '--i18n', '-f', '--format', '-o', '--out',
48
49
  '-t', '--tags', '-n', '--name', '-e', '--exclude',
49
- PROFILE_SHORT_FLAG, PROFILE_LONG_FLAG,
50
+ PROFILE_SHORT_FLAG, PROFILE_LONG_FLAG, RETRY_FLAG,
50
51
  '-l', '--lines', '--port',
51
52
  '-I', '--snippet-type']
52
53
  ORDER_TYPES = %w{defined random}
@@ -188,6 +189,9 @@ module Cucumber
188
189
  "Disables all profile loading to avoid using the 'default' profile.") do |v|
189
190
  @disable_profile_loading = true
190
191
  end
192
+ opts.on("#{RETRY_FLAG} ATTEMPTS", "Specify the number of times to retry failing tests (default: 0)") do |v|
193
+ @options[:retry] = v.to_i
194
+ end
191
195
  opts.on("-c", "--[no-]color",
192
196
  "Whether or not to use ANSI color in the output. Cucumber decides",
193
197
  "based on your platform and the output destination if not specified.") do |v|
@@ -450,7 +454,8 @@ TEXT
450
454
  :diff_enabled => true,
451
455
  :snippets => true,
452
456
  :source => true,
453
- :duration => true
457
+ :duration => true,
458
+ :retry => 0
454
459
  }
455
460
  end
456
461
  end
@@ -16,7 +16,7 @@ module Cucumber
16
16
  end
17
17
 
18
18
  def features
19
- lines.map { |l| l.scan(/(?:^| )(.*?\.feature(?:(?::\d+)*))/) }.flatten
19
+ lines.map { |l| l.scan(/(?:^| |)(.*?\.feature(?:(?::\d+)*))/) }.flatten
20
20
  end
21
21
 
22
22
  private
@@ -65,6 +65,10 @@ module Cucumber
65
65
  @options[:fail_fast]
66
66
  end
67
67
 
68
+ def retry_attempts
69
+ @options[:retry]
70
+ end
71
+
68
72
  def guess?
69
73
  @options[:guess]
70
74
  end
@@ -0,0 +1,9 @@
1
+ module Cucumber
2
+ module Events
3
+
4
+ # Event fired after aall test cases have finished executing
5
+ class FinishedTesting
6
+ end
7
+
8
+ end
9
+ end
@@ -1,3 +1,5 @@
1
+ require 'digest/sha2'
2
+
1
3
  module Cucumber
2
4
  module Filters
3
5
 
@@ -29,7 +31,10 @@ module Cucumber
29
31
  private
30
32
 
31
33
  def shuffled_test_cases
32
- @test_cases.shuffle(random: Random.new(seed))
34
+ digester = Digest::SHA2.new(256)
35
+ @test_cases.map.with_index.
36
+ sort_by { |_, index| digester.digest((@seed + index).to_s) }.
37
+ map { |test_case, _| test_case }
33
38
  end
34
39
 
35
40
  attr_reader :seed
@@ -0,0 +1,32 @@
1
+ require 'cucumber/core/filter'
2
+ require 'cucumber/running_test_case'
3
+ require 'cucumber/events/bus'
4
+ require 'cucumber/events/after_test_case'
5
+
6
+ module Cucumber
7
+ module Filters
8
+ class Retry < Core::Filter.new(:configuration)
9
+
10
+ def test_case(test_case)
11
+ configuration.on_event(:after_test_case) do |event|
12
+ next unless retry_required?(test_case, event)
13
+
14
+ test_case_counts[test_case] += 1
15
+ event.test_case.describe_to(receiver)
16
+ end
17
+
18
+ super
19
+ end
20
+
21
+ private
22
+
23
+ def retry_required?(test_case, event)
24
+ event.test_case == test_case && event.result.failed? && test_case_counts[test_case] < configuration.retry_attempts
25
+ end
26
+
27
+ def test_case_counts
28
+ @test_case_counts ||= Hash.new {|h,k| h[k] = 0 }
29
+ end
30
+ end
31
+ end
32
+ end