cucumber 1.3.10 → 2.0.0

data/features/docs/formatters/junit_formatter.feature

@@ -0,0 +1,316 @@
+@spawn
+Feature: JUnit output formatter
+  In order for developers to create test reports with ant
+  Cucumber should be able to output JUnit xml files
+  
+  Background:
+    Given the standard step definitions
+    And a file named "features/one_passing_one_failing.feature" with:
+      """
+      Feature: One passing scenario, one failing scenario
+
+        Scenario: Passing
+          Given this step passes
+
+        Scenario: Failing
+          Given this step fails
+      """
+    And a file named "features/some_subdirectory/one_passing_one_failing.feature" with:
+      """
+      Feature: Subdirectory - One passing scenario, one failing scenario
+
+        Scenario: Passing
+          Given this step passes
+
+        Scenario: Failing
+          Given this step fails
+      """
+    And a file named "features/pending.feature" with:
+      """
+      Feature: Pending step
+
+        Scenario: Pending
+          Given this step is pending
+
+        Scenario: Undefined
+          Given this step is undefined
+      """
+    And a file named "features/pending.feature" with:
+      """
+      Feature: Pending step
+
+        Scenario: Pending
+          Given this step is pending
+
+        Scenario: Undefined
+          Given this step is undefined
+      """
+    And a file named "features/scenario_outline.feature" with:
+      """
+      Feature: Scenario outlines
+
+        Scenario Outline: Using scenario outlines
+          Given this step <type>
+
+          Examples:
+            | type   |
+            | passes |
+            | fails  |
+      """
+
+  Scenario: one feature, one passing scenario, one failing scenario
+    When I run `cucumber --format junit --out tmp/ features/one_passing_one_failing.feature`
+    Then it should fail with:
+      """
+
+      """
+    And the junit output file "tmp/TEST-features-one_passing_one_failing.xml" should contain:
+      """
+      <?xml version="1.0" encoding="UTF-8"?>
+      <testsuite failures="1" errors="0" skipped="0" tests="2" time="0.05" name="One passing scenario, one failing scenario">
+      <testcase classname="One passing scenario, one failing scenario" name="Passing" time="0.05">
+        <system-out/>
+        <system-err/>
+      </testcase>
+      <testcase classname="One passing scenario, one failing scenario" name="Failing" time="0.05">
+        <failure message="failed Failing" type="failed">
+          <![CDATA[Scenario: Failing
+
+      Given this step fails
+
+      Message:
+	]]>
+          <![CDATA[ (RuntimeError)
+	./features/step_definitions/steps.rb:4:in `/^this step fails$/'
+	features/one_passing_one_failing.feature:7:in `Given this step fails']]>
+        </failure>
+        <system-out/>
+        <system-err/>
+      </testcase>
+        <system-out>
+          <![CDATA[]]>
+        </system-out>
+        <system-err>
+          <![CDATA[]]>
+        </system-err>
+      </testsuite>
+
+      """
+
+  Scenario: one feature in a subdirectory, one passing scenario, one failing scenario
+    When I run `cucumber --format junit --out tmp/ features/some_subdirectory/one_passing_one_failing.feature --require features`
+    Then it should fail with:
+      """
+
+      """
+    And the junit output file "tmp/TEST-features-some_subdirectory-one_passing_one_failing.xml" should contain:
+      """
+      <?xml version="1.0" encoding="UTF-8"?>
+      <testsuite failures="1" errors="0" skipped="0" tests="2" time="0.05" name="Subdirectory - One passing scenario, one failing scenario">
+      <testcase classname="Subdirectory - One passing scenario, one failing scenario" name="Passing" time="0.05">
+        <system-out/>
+        <system-err/>
+      </testcase>
+      <testcase classname="Subdirectory - One passing scenario, one failing scenario" name="Failing" time="0.05">
+        <failure message="failed Failing" type="failed">
+          <![CDATA[Scenario: Failing
+
+      Given this step fails
+
+      Message:
+	]]>
+          <![CDATA[ (RuntimeError)
+	./features/step_definitions/steps.rb:4:in `/^this step fails$/'
+	features/some_subdirectory/one_passing_one_failing.feature:7:in `Given this step fails']]>
+        </failure>
+        <system-out/>
+        <system-err/>
+      </testcase>
+        <system-out>
+          <![CDATA[]]>
+        </system-out>
+        <system-err>
+          <![CDATA[]]>
+        </system-err>
+      </testsuite>
+
+      """
+
+  Scenario: pending and undefined steps are reported as skipped
+    When I run `cucumber --format junit --out tmp/ features/pending.feature`
+    Then it should pass with:
+      """
+      
+      """
+    And the junit output file "tmp/TEST-features-pending.xml" should contain:
+      """
+      <?xml version="1.0" encoding="UTF-8"?>
+      <testsuite failures="0" errors="0" skipped="2" tests="2" time="0.05" name="Pending step">
+      <testcase classname="Pending step" name="Pending" time="0.05">
+        <skipped/>
+        <system-out/>
+        <system-err/>
+      </testcase>
+      <testcase classname="Pending step" name="Undefined" time="0.05">
+        <skipped/>
+        <system-out/>
+        <system-err/>
+      </testcase>
+        <system-out>
+          <![CDATA[]]>
+        </system-out>
+        <system-err>
+          <![CDATA[]]>
+        </system-err>
+      </testsuite>
+      
+      """
+
+  Scenario: pending and undefined steps with strict option should fail
+    When I run `cucumber --format junit --out tmp/ features/pending.feature --strict`
+    Then it should fail with:
+      """
+
+      """
+    And the junit output file "tmp/TEST-features-pending.xml" should contain:
+      """
+      <?xml version="1.0" encoding="UTF-8"?>
+      <testsuite failures="2" errors="0" skipped="0" tests="2" time="0.05" name="Pending step">
+      <testcase classname="Pending step" name="Pending" time="0.05">
+        <failure message="pending Pending" type="pending">
+          <![CDATA[Scenario: Pending
+
+      Given this step is pending
+
+      Message:
+      ]]>
+          <![CDATA[TODO (Cucumber::Pending)
+      ./features/step_definitions/steps.rb:3:in `/^this step is pending$/'
+      features/pending.feature:4:in `Given this step is pending']]>
+        </failure>
+        <system-out/>
+        <system-err/>
+      </testcase>
+      <testcase classname="Pending step" name="Undefined" time="0.05">
+        <failure message="undefined Undefined" type="undefined">
+          <![CDATA[Scenario: Undefined
+      
+      Given this step is undefined
+
+      Message:
+      ]]>
+          <![CDATA[Undefined step: "this step is undefined" (Cucumber::Undefined)
+      features/pending.feature:7:in `Given this step is undefined']]>
+        </failure>
+        <system-out/>
+        <system-err/>
+      </testcase>
+        <system-out>
+          <![CDATA[]]>
+        </system-out>
+        <system-err>
+          <![CDATA[]]>
+        </system-err>
+      </testsuite>
+
+      """
+    
+  Scenario: run all features
+    When I run `cucumber --format junit --out tmp/ features`
+    Then it should fail with:
+      """
+      
+      """
+    And a file named "tmp/TEST-features-one_passing_one_failing.xml" should exist
+    And a file named "tmp/TEST-features-pending.xml" should exist
+  
+  Scenario: show correct error message if no --out is passed
+    When I run `cucumber --format junit features`
+    Then the stderr should not contain:
+      """
+can't convert .* into String \(TypeError\)
+      """
+    And the stderr should contain:
+      """
+You *must* specify --out DIR for the junit formatter
+      """
+
+  Scenario: one feature, one scenario outline, two examples: one passing, one failing
+    When I run `cucumber --format junit --out tmp/ features/scenario_outline.feature`
+    Then it should fail with:
+      """
+
+      """
+    And the junit output file "tmp/TEST-features-scenario_outline.xml" should contain:
+      """
+      <?xml version="1.0" encoding="UTF-8"?>
+      <testsuite failures="1" errors="0" skipped="0" tests="2" time="0.05" name="Scenario outlines">
+      <testcase classname="Scenario outlines" name="Using scenario outlines (outline example : | passes |)" time="0.05">
+        <system-out/>
+        <system-err/>
+      </testcase>
+      <testcase classname="Scenario outlines" name="Using scenario outlines (outline example : | fails |)" time="0.05">
+        <failure message="failed Using scenario outlines (outline example : | fails |)" type="failed">
+          <![CDATA[Scenario Outline: Using scenario outlines
+      
+      Example row: | fails |
+      
+      Message:
+      ]]>
+          <![CDATA[ (RuntimeError)
+      ./features/step_definitions/steps.rb:4:in `/^this step fails$/'
+      features/scenario_outline.feature:9:in `Given this step fails'
+      features/scenario_outline.feature:4:in `Given this step <type>']]>
+        </failure>
+        <system-out/>
+        <system-err/>
+      </testcase>
+        <system-out>
+          <![CDATA[]]>
+        </system-out>
+        <system-err>
+          <![CDATA[]]>
+        </system-err>
+      </testsuite>
+
+      """ 
+
+  Scenario: one feature, one scenario outline, two examples: one passing, one failing with --expand option
+    When I run `cucumber --expand --format junit --out tmp/ features/scenario_outline.feature`
+    Then it should fail with exactly:
+      """
+
+      """
+    And the junit output file "tmp/TEST-features-scenario_outline.xml" should contain:
+      """
+      <?xml version="1.0" encoding="UTF-8"?>
+      <testsuite failures="1" errors="0" skipped="0" tests="2" time="0.05" name="Scenario outlines">
+      <testcase classname="Scenario outlines" name="Using scenario outlines (outline example : | passes |)" time="0.05">
+        <system-out/>
+        <system-err/>
+      </testcase>
+      <testcase classname="Scenario outlines" name="Using scenario outlines (outline example : | fails |)" time="0.05">
+        <failure message="failed Using scenario outlines (outline example : | fails |)" type="failed">
+          <![CDATA[Scenario Outline: Using scenario outlines
+      
+      Example row: | fails |
+      
+      Message:
+      ]]>
+          <![CDATA[ (RuntimeError)
+      ./features/step_definitions/steps.rb:4:in `/^this step fails$/'
+      features/scenario_outline.feature:9:in `Given this step fails'
+      features/scenario_outline.feature:4:in `Given this step <type>']]>
+        </failure>
+        <system-out/>
+        <system-err/>
+      </testcase>
+        <system-out>
+          <![CDATA[]]>
+        </system-out>
+        <system-err>
+          <![CDATA[]]>
+        </system-err>
+      </testsuite>
+
+      """ 

data/features/docs/formatters/pretty_formatter.feature

@@ -0,0 +1,73 @@
+Feature: Pretty output formatter
+
+  Background:
+    Given a file named "features/scenario_outline_with_undefined_steps.feature" with:
+      """
+      Feature:
+
+        Scenario Outline:
+          Given this step is undefined
+
+        Examples:
+          |foo|
+          |bar|
+      """
+
+  Scenario: an scenario outline, one undefined step, one random example, expand flag on
+    When I run `cucumber features/scenario_outline_with_undefined_steps.feature --format pretty --expand `
+    Then it should pass
+
+  Scenario: when using a profile the output should include 'Using the default profile...'
+    And a file named "cucumber.yml" with:
+    """
+      default: -r features
+    """
+    When I run `cucumber --profile default --format pretty`
+    Then it should pass
+    And the output should contain:
+    """
+    Using the default profile...
+    """
+  Scenario: Hook output should be printed before hook exception
+    Given the standard step definitions
+    And a file named "features/test.feature" with:
+      """
+      Feature:
+        Scenario:
+          Given this step passes
+      """
+    And a file named "features/step_definitions/output_steps.rb" with:
+      """
+      Before do
+        puts "Before hook"
+       end
+
+      AfterStep do
+        puts "AfterStep hook"
+      end
+
+      After do
+        puts "After hook"
+	raise "error"
+      end
+      """
+    When I run `cucumber -q -f pretty features/test.feature`
+    Then the stderr should not contain anything
+    Then it should fail with:
+      """
+      Feature: 
+ 
+        Scenario: 
+            Before hook
+          Given this step passes
+            AfterStep hook
+            After hook
+            error (RuntimeError)
+            ./features/step_definitions/output_steps.rb:11:in `After'
+      
+      Failing Scenarios:
+      cucumber features/test.feature:2
+      
+      1 scenario (1 failed)
+      1 step (1 passed)
+      """

data/features/{progress_formatter.feature → docs/formatters/progress_formatter.feature}

@@ -6,7 +6,7 @@ Feature: Progress output formatter
       Feature:
 
         Scenario Outline:
-          Given an undefined step
+          Given this step is undefined
 
         Examples:
           |foo|

data/features/docs/formatters/rerun_formatter.feature

@@ -0,0 +1,136 @@
+Feature: Rerun formatter
+
+  The rerun formatter writes an output that's perfect for
+  passing to Cucumber when you want to rerun only the 
+  scenarios that have failed.
+
+  You can save off the rerun output to a file by using it like this:
+
+  `cucumber -f rerun --out .cucumber.rerun`
+
+  Now you can pass that file's content to Cucumber to tell it
+  which scenarios to run:
+
+  `cucumber \`cat .cucumber.rerun\``
+
+  This is useful when debugging in a large suite of features.
+
+  Background:
+    Given the standard step definitions
+
+  Scenario: Regular scenarios
+    Given a file named "features/mixed.feature" with:
+      """
+      Feature: Mixed
+
+        Scenario:
+          Given this step fails
+
+        Scenario:
+          Given this step is undefined
+
+        Scenario:
+          Given this step is pending
+
+        Scenario:
+          Given this step passes
+
+      """
+    And a file named "features/all_good.feature" with:
+      """
+      Feature: All good
+
+        Scenario:
+          Given this step passes
+      """
+
+    When I run `cucumber -f rerun`
+    Then it should fail with:
+      """
+      features/mixed.feature:3:6:9
+      """
+
+  Scenario: Scenario outlines
+    For details see https://github.com/cucumber/cucumber/issues/57
+    Given a file named "features/one_passing_one_failing.feature" with:
+      """
+      Feature: One passing example, one failing example
+
+        Scenario Outline:
+          Given this step <status>
+        
+        Examples:
+          | status |
+          | passes |
+          | fails  |
+
+      """
+    When I run `cucumber -f rerun`
+    Then it should fail with:
+    """
+    features/one_passing_one_failing.feature:9
+    """
+
+  Scenario: Failing background
+    Given a file named "features/failing_background.feature" with:
+      """
+      Feature: Failing background sample
+
+        Background:
+          Given this step fails
+
+        Scenario: failing background
+          Then this step passes
+
+        Scenario: another failing background
+          Then this step passes
+      """
+    When I run `cucumber -f rerun`
+    Then it should fail with:
+    """
+    features/failing_background.feature:6:9
+    """
+
+  Scenario: Failing background with scenario outline
+    Given a file named "features/failing_background_outline.feature" with:
+      """
+      Feature: Failing background sample with scenario outline
+
+        Background:
+          Given this step fails
+
+        Scenario Outline:
+          Then this step <status>
+
+        Examples:
+          | status |
+          | passes |
+          | passes |
+      """
+    When I run `cucumber features/failing_background_outline.feature -r features -f rerun`
+    Then it should fail with:
+    """
+    features/failing_background_outline.feature:11:12
+    """
+
+  Scenario: Scenario outlines with expand
+    For details see https://github.com/cucumber/cucumber/issues/503
+
+    Given a file named "features/one_passing_one_failing.feature" with:
+    """
+      Feature: One passing example, one failing example
+
+        Scenario Outline:
+          Given this step <status>
+
+        Examples:
+          | status |
+          | passes |
+          | fails  |
+
+      """
+    When I run `cucumber --expand -f rerun`
+    Then it should fail with:
+    """
+    features/one_passing_one_failing.feature:9
+    """

data/features/docs/formatters/usage_formatter.feature

@@ -0,0 +1,102 @@
+Feature: Usage formatter
+
+  In order to see where step definitions are used
+  Developers should be able to see a list of step definitions and their use
+
+  Background:
+    Given a file named "features/f.feature" with:
+      """
+      Feature: F
+        Background: A
+          Given A
+        Scenario: B
+          Given B
+        Scenario Outline: CA
+          Given <x>
+          And B
+          Examples:
+            |x|
+            |C|
+            |A|
+        Scenario: AC
+          Given A
+          Given C
+      """
+    And a file named "features/step_definitions/steps.rb" with:
+      """
+      Given(/A/) { }
+      Given(/B/) { }
+      Given(/C/) { }
+      Given(/D/) { }
+      """
+
+  @wip-new-core
+  Scenario: Run with --format usage
+    When I run `cucumber -f usage --dry-run`
+    Then it should pass with exactly:
+      """
+      --------
+      
+      /A/       # features/step_definitions/steps.rb:1
+        Given A # features/f.feature:3
+        Given A # features/f.feature:12
+        Given A # features/f.feature:14
+      /B/       # features/step_definitions/steps.rb:2
+        Given B # features/f.feature:5
+        And B   # features/f.feature:11
+        And B   # features/f.feature:12
+      /C/       # features/step_definitions/steps.rb:3
+        Given C # features/f.feature:11
+        Given C # features/f.feature:15
+      /D/       # features/step_definitions/steps.rb:4
+        NOT MATCHED BY ANY STEPS
+      
+      4 scenarios (4 skipped)
+      11 steps (11 skipped)
+      0m0.012s
+
+      """
+
+  Scenario: Run with --expand --format usage
+    When I run `cucumber -x -f usage --dry-run`
+    Then it should pass with exactly:
+      """
+      ----------
+      
+      /A/       # features/step_definitions/steps.rb:1
+        Given A # features/f.feature:3
+        Given A # features/f.feature:12
+        Given A # features/f.feature:14
+      /B/       # features/step_definitions/steps.rb:2
+        Given B # features/f.feature:5
+        And B   # features/f.feature:11
+        And B   # features/f.feature:12
+      /C/       # features/step_definitions/steps.rb:3
+        Given C # features/f.feature:11
+        Given C # features/f.feature:15
+      /D/       # features/step_definitions/steps.rb:4
+        NOT MATCHED BY ANY STEPS
+      
+      4 scenarios (4 skipped)
+      11 steps (11 skipped)
+      0m0.012s
+
+      """
+
+    Scenario: Run with --format stepdefs
+      When I run `cucumber -f stepdefs --dry-run`
+      Then it should pass with exactly:
+        """
+        --------
+        
+        /A/   # features/step_definitions/steps.rb:1
+        /B/   # features/step_definitions/steps.rb:2
+        /C/   # features/step_definitions/steps.rb:3
+        /D/   # features/step_definitions/steps.rb:4
+          NOT MATCHED BY ANY STEPS
+        
+        4 scenarios (4 skipped)
+        11 steps (11 skipped)
+        0m0.012s
+
+        """

data/features/docs/getting_started.feature

@@ -0,0 +1,27 @@
+Feature: Getting started
+
+  To get started, just open a command prompt in an empty directory and run 
+  `cucumber`. You'll be prompted for what to do next.
+
+  @spawn
+  Scenario: Run Cucumber in an empty directory
+    Given a directory without standard Cucumber project directory structure
+    When I run `cucumber`
+    Then it should fail with:
+      """
+      No such file or directory - features. You can use `cucumber --init` to get started.
+      """
+
+  Scenario: Accidentally run Cucumber in a folder with Ruby files in it.
+    Given a directory without standard Cucumber project directory structure
+    And a file named "should_not_load.rb" with:
+      """
+      puts 'this will not be shown'
+      """
+    When I run `cucumber`
+    Then the exit status should be 1
+    And the output should not contain:
+      """
+      this will not be shown
+
+      """