busser-behave 0.1.3

data/vendor/behave/features/runner.abort_by_user.feature

@@ -0,0 +1,305 @@
+Feature: Test run can be aborted by the user
+
+  As a tester
+  I want sometimes to abort a test run (because it is anyway failing, etc.)
+  So that I am more productive.
+
+  . NOTES:
+  .  * The test runner should fail gracefully (most of the times)
+  .  * At least some cleanup hooks should be called (in general)
+
+
+  @setup
+  Scenario: Feature Setup
+    Given a new working directory
+    And a file named "features/steps/aborting_steps.py" with:
+        """
+        from behave import step
+
+        @step('{word:w} step passes')
+        def step_passes(context, word):
+            pass
+
+        @step('the user aborts the test run')
+        def step_user_aborts_testrun(context):
+            raise KeyboardInterrupt()
+        """
+    And a file named "features/scenarios_pass3.feature" with:
+        """
+        Feature:
+          Scenario:
+            Given a step passes
+            When another step passes
+
+          Scenario:
+            Given first step passes
+            When second step passes
+            Then third step passes
+
+          Scenario:
+            Then last step passes
+        """
+
+
+  Scenario: Abort test run in step definition
+    Given a file named "features/aborting_in_step.feature" with:
+        """
+        Feature: User aborts test run in a step definition
+          Scenario:
+            Given a step passes
+            When another step passes
+
+          Scenario: User aborts here
+            Given first step passes
+            When the user aborts the test run
+            Then third step passes
+
+          Scenario:
+            Then last step passes
+        """
+    And an empty file named "features/environment.py"
+    When I run "behave -f plain -T features/aborting_in_step.feature"
+    Then it should fail with:
+        """
+        0 features passed, 1 failed, 0 skipped
+        1 scenario passed, 1 failed, 0 skipped, 1 untested
+        3 steps passed, 1 failed, 1 skipped, 0 undefined, 1 untested
+        """
+    And the command output should contain:
+        """
+        Feature: User aborts test run in a step definition
+
+        Scenario:
+          Given a step passes ... passed
+          When another step passes ... passed
+
+        Scenario: User aborts here
+          Given first step passes ... passed
+          When the user aborts the test run ... failed
+        ABORTED: By user (KeyboardInterrupt).
+
+        ABORTED: By user.
+
+        Failing scenarios:
+          features/aborting_in_step.feature:6  User aborts here
+        """
+    But note that "the last scenario is untested (not-run) due to the user abort"
+
+
+  Scenario: Abort test run in before_scenario hook
+    Given a file named "features/aborting_in_before_scenario_hook.feature" with:
+        """
+        Feature: User aborts test run in before_scenario hook of S2
+          Scenario: S1
+            Given a step passes
+            When another step passes
+
+          @user.aborts.before_scenario
+          Scenario: S2 -- User aborts here
+            Given first step passes
+            When second step passes
+            Then third step passes
+
+          Scenario: S3
+            Then last step passes
+        """
+    And a file named "features/environment.py" with:
+        """
+        def before_scenario(context, scenario):
+            if "user.aborts.before_scenario" in scenario.tags:
+                user_aborts_testrun_here()
+
+        def user_aborts_testrun_here():
+            raise KeyboardInterrupt()
+        """
+    When I run "behave -f plain -T features/aborting_in_before_scenario_hook.feature"
+    Then it should fail with:
+        """
+        ABORTED: By user.
+        0 features passed, 1 failed, 0 skipped
+        1 scenario passed, 0 failed, 0 skipped, 2 untested
+        2 steps passed, 0 failed, 0 skipped, 0 undefined, 4 untested
+        """
+    And the command output should contain:
+        """
+        Feature: User aborts test run in before_scenario hook of S2
+
+          Scenario: S1
+            Given a step passes ... passed
+            When another step passes ... passed
+        """
+    But the command output should not contain "Scenario: S2 -- User aborts here"
+    But the command output should not contain "Scenario: S3"
+    And note that "the second and third scenario is not run"
+
+
+  Scenario: Abort test run in after_scenario hook
+    Given a file named "features/aborting_in_after_scenario_hook.feature" with:
+        """
+        Feature: User aborts test run in after_scenario hook
+          Scenario:
+            Given a step passes
+            When another step passes
+
+          @user.aborts.after_scenario
+          Scenario: User aborts here
+            Given first step passes
+            When second step passes
+            Then third step passes
+
+          Scenario:
+            Then last step passes
+        """
+    And a file named "features/environment.py" with:
+        """
+        def after_scenario(context, scenario):
+            if "user.aborts.after_scenario" in scenario.tags:
+                raise KeyboardInterrupt()
+        """
+    When I run "behave -f plain -T features/aborting_in_after_scenario_hook.feature"
+    Then it should fail with:
+        """
+        ABORTED: By user.
+        0 features passed, 1 failed, 0 skipped
+        2 scenarios passed, 0 failed, 0 skipped, 1 untested
+        5 steps passed, 0 failed, 0 skipped, 0 undefined, 1 untested
+        """
+    And the command output should contain:
+        """
+        Feature: User aborts test run in after_scenario hook
+
+          Scenario:
+            Given a step passes ... passed
+            When another step passes ... passed
+
+          Scenario: User aborts here
+            Given first step passes ... passed
+            When second step passes ... passed
+            Then third step passes ... passed
+        """
+    But the command output should not contain:
+        """
+          Scenario:
+            Then last step passes ... passed
+        """
+    And note that "the last scenario is not run"
+
+
+  Scenario: Abort test run in before_feature hook
+    Given a file named "features/aborting_in_before_feature_hook.feature" with:
+        """
+        @user.aborts.before_feature
+        Feature: User aborts test HERE
+          Scenario:
+            Given a step passes
+            When another step passes
+
+          Scenario:
+            Given first step passes
+            When second step passes
+            Then third step passes
+
+          Scenario:
+            Then last step passes
+        """
+    And a file named "features/environment.py" with:
+        """
+        from __future__ import print_function
+
+        def before_feature(context, feature):
+            if "user.aborts.before_feature" in feature.tags:
+                print("ABORTED in before_feature: %s" % feature.location)
+                raise KeyboardInterrupt()
+        """
+    When I run "behave -f plain -T features/aborting_in_before_feature_hook.feature"
+    Then it should fail with:
+        """
+        ABORTED in before_feature: features/aborting_in_before_feature_hook.feature:2
+
+        ABORTED: By user.
+        0 features passed, 0 failed, 0 skipped, 1 untested
+        0 scenarios passed, 0 failed, 0 skipped, 3 untested
+        0 steps passed, 0 failed, 0 skipped, 0 undefined, 6 untested
+        """
+    But note that "the feature is not run"
+    And note that "the formatters are not informed of this feature"
+
+
+  Scenario: Abort test run in after_feature hook
+    Given a file named "features/aborting_in_after_feature_hook.feature" with:
+        """
+        @user.aborts.after_feature
+        Feature: User aborts test after HERE
+          Scenario:
+            Given a step passes
+            When another step passes
+
+          Scenario:
+            Given first step passes
+            When second step passes
+            Then third step passes
+
+          Scenario:
+            Then last step passes
+        """
+    And a file named "features/environment.py" with:
+        """
+        def after_feature(context, feature):
+            if "user.aborts.after_feature" in feature.tags:
+                raise KeyboardInterrupt()
+        """
+    When I run "behave -f plain -T features/aborting_in_after_feature_hook.feature"
+    Then it should fail with:
+        """
+        ABORTED: By user.
+        1 feature passed, 0 failed, 0 skipped
+        3 scenarios passed, 0 failed, 0 skipped
+        6 steps passed, 0 failed, 0 skipped, 0 undefined
+        """
+    But note that "the behave command fails, but all features/scenarios passed"
+
+
+  Scenario: Abort test run in before_all hook
+
+    Note that this situation is not handled very gracefully (yet).
+
+      Given a file named "features/scenarios_pass3.feature" exists
+      And a file named "features/environment.py" with:
+        """
+        def before_all(context):
+            raise KeyboardInterrupt()   #< ABORT-HERE
+        """
+      When I run "behave -f plain -T features/scenarios_pass3.feature"
+      Then it should fail with:
+        """
+        Traceback (most recent call last):
+        """
+      And the command output should contain:
+        """
+        File "features/environment.py", line 2, in before_all
+          raise KeyboardInterrupt()   #< ABORT-HERE
+        """
+      And note that "no feature is not run"
+
+
+  Scenario: Abort test run in after_all hook
+
+    Note that this situation is not handled very gracefully (yet).
+
+      Given a file named "features/scenarios_pass3.feature" exists
+      And a file named "features/environment.py" with:
+        """
+        def after_all(context):
+            raise KeyboardInterrupt()   #< ABORT-HERE
+        """
+      When I run "behave -f plain -T features/scenarios_pass3.feature"
+      Then it should fail with:
+        """
+        Traceback (most recent call last):
+        """
+      And the command output should contain:
+        """
+        File "features/environment.py", line 2, in after_all
+          raise KeyboardInterrupt()   #< ABORT-HERE
+        """
+      And note that "all features are run"

data/vendor/behave/features/runner.continue_after_failed_step.feature

@@ -0,0 +1,136 @@
+Feature: Runner should continue after a failed step
+
+    As a tester or test writer
+    I want that the remaining scenario steps are executed even when a step fails
+    So that I see all remaining failures.
+
+    . NOTES:
+    .  This may cause a number of correlated failures in the remaining steps.
+    .  Therefore the default behavior of the runner is to skip the
+    .  remaining scenario steps after a failure has occurred.
+    .
+    . RELATED TO: CR #299 (duplicated: #314)
+
+    @setup
+    Scenario: Test Setup
+        Given a new working directory
+        And a file named "features/steps/steps.py" with:
+            """
+            from behave import step
+            import sys
+
+            @step('{word:w} step passes')
+            def step_passes(context, word):
+                pass
+
+            @step('{word:w} step fails')
+            def step_fails(context, word):
+                assert False, "XFAIL (in: %s step)" % word
+            """
+        And a file named "features/alice.feature" with:
+            """
+            Feature: Alice
+              Scenario: Passing
+                Given first step passes
+                When second step passes
+                Then third step passes
+
+              Scenario: Fails in second step
+                Given a step passes
+                When second step fails
+                Then another step passes
+
+              @runner.continue_after_failed_step
+              Scenario: Fails in first and third step
+                Given first step fails
+                When second step passes
+                Then third step fails
+            """
+
+
+    Scenario: Runner continues after failed step in all scenarios
+      Given a file named "features/environment.py" with:
+          """
+          from behave.model import Scenario
+
+          def before_all(context):
+              userdata = context.config.userdata
+              continue_after_failed = userdata.getbool("runner.continue_after_failed_step", False)
+              Scenario.continue_after_failed_step = continue_after_failed
+          """
+      And a file named "behave.ini" with:
+          """
+          [behave]
+          show_timings = false
+
+          [behave.userdata]
+          runner.continue_after_failed_step = true
+          """
+      When I run "behave -f plain features/alice.feature"
+      Then it should fail with:
+          """
+          Failing scenarios:
+            features/alice.feature:7  Fails in second step
+            features/alice.feature:13  Fails in first and third step
+
+          0 features passed, 1 failed, 0 skipped
+          1 scenario passed, 2 failed, 0 skipped
+          6 steps passed, 3 failed, 0 skipped, 0 undefined
+          """
+      And the command output should contain:
+          """
+          Scenario: Fails in second step
+              Given a step passes ... passed
+              When second step fails ... failed
+                Assertion Failed: XFAIL (in: second step)
+              Then another step passes ... passed
+
+          Scenario: Fails in first and third step
+            Given first step fails ... failed
+              Assertion Failed: XFAIL (in: first step)
+            When second step passes ... passed
+            Then third step fails ... failed
+              Assertion Failed: XFAIL (in: third step)
+          """
+      But note that "step execution continues after failed step(s)"
+      And note that "no steps are skipped"
+
+
+    Scenario: Runner continues after failed step in some scenarios
+
+      Enable this feature only on tagged scenarios (or features).
+
+      Given a file named "features/environment.py" with:
+          """
+          def before_scenario(context, scenario):
+              if "runner.continue_after_failed_step" in scenario.effective_tags:
+                  scenario.continue_after_failed_step = True
+          """
+      When I run "behave -f plain -T features/alice.feature"
+      Then it should fail with:
+          """
+          Failing scenarios:
+            features/alice.feature:7  Fails in second step
+            features/alice.feature:13  Fails in first and third step
+
+          0 features passed, 1 failed, 0 skipped
+          1 scenario passed, 2 failed, 0 skipped
+          5 steps passed, 3 failed, 1 skipped, 0 undefined
+          """
+      And the command output should contain:
+          """
+          Scenario: Fails in second step
+            Given a step passes ... passed
+            When second step fails ... failed
+              Assertion Failed: XFAIL (in: second step)
+
+          Scenario: Fails in first and third step
+            Given first step fails ... failed
+              Assertion Failed: XFAIL (in: first step)
+            When second step passes ... passed
+            Then third step fails ... failed
+              Assertion Failed: XFAIL (in: third step)
+          """
+      But note that "step execution continues after failed step in tagged scenario only"
+      And note that "some steps are skipped (in 2nd, untagged scenario)"
+

data/vendor/behave/features/runner.default_format.feature

@@ -0,0 +1,175 @@
+@sequential
+Feature: Default Formatter
+
+  . SPECIFICATION:
+  .  * Default formatter is used when no other formatter is specified/provided.
+  .  * Default formatter uses stdout as default output/outfile.
+  .  * Pretty formatter is the default formatter.
+  .  * Behave configfile can specify the default formatter.
+
+
+    @setup
+    Scenario: Test Setup
+      Given a new working directory
+      And a file named "features/steps/passing_steps.py" with:
+        """
+        from behave import step
+
+        @step('a step passes')
+        def step_passes(context):
+            pass
+        """
+      And a file named "features/alice.feature" with:
+        """
+        Feature: Alice
+          Scenario: A1
+            Given a step passes
+            When a step passes
+            Then a step passes
+        """
+      And a file named "features/bob.feature" with:
+        """
+        Feature: Bob
+          Scenario: B1
+            When a step passes
+            Then a step passes
+        """
+
+    @no_configfile
+    Scenario: Pretty formatter is used as default formatter if no other is defined
+      Given a file named "behave.ini" does not exist
+      When I run "behave -c features/"
+      Then it should pass with:
+        """
+        2 features passed, 0 failed, 0 skipped
+        2 scenarios passed, 0 failed, 0 skipped
+        """
+      And the command output should contain:
+        """
+        Feature: Alice # features/alice.feature:1
+          Scenario: A1          # features/alice.feature:2
+            Given a step passes # features/steps/passing_steps.py:3
+            When a step passes  # features/steps/passing_steps.py:3
+            Then a step passes  # features/steps/passing_steps.py:3
+
+        Feature: Bob # features/bob.feature:1
+          Scenario: B1         # features/bob.feature:2
+            When a step passes # features/steps/passing_steps.py:3
+            Then a step passes # features/steps/passing_steps.py:3
+        """
+
+
+    @with_configfile
+    Scenario: Configfile can define own default formatter
+      Given a file named "behave.ini" with:
+        """
+        [behave]
+        default_format = plain
+        show_timings = false
+        """
+      When I run "behave features/"
+      Then it should pass with:
+        """
+        2 features passed, 0 failed, 0 skipped
+        2 scenarios passed, 0 failed, 0 skipped
+        """
+      And the command output should contain:
+        """
+        Feature: Alice
+          Scenario: A1
+            Given a step passes ... passed
+            When a step passes ... passed
+            Then a step passes ... passed
+
+        Feature: Bob
+          Scenario: B1
+            When a step passes ... passed
+            Then a step passes ... passed
+        """
+
+    @with_configfile
+    Scenario: Use default formatter with own outfile instead of stdout
+      Given a file named "behave.ini" with:
+        """
+        [behave]
+        default_format = plain
+        show_timings = false
+        """
+      When I run "behave --outfile=output/default.out features/"
+      Then it should pass with:
+        """
+        2 features passed, 0 failed, 0 skipped
+        2 scenarios passed, 0 failed, 0 skipped
+        """
+      And the file "output/default.out" should contain:
+        """
+        Feature: Alice
+          Scenario: A1
+            Given a step passes ... passed
+            When a step passes ... passed
+            Then a step passes ... passed
+
+        Feature: Bob
+          Scenario: B1
+            When a step passes ... passed
+            Then a step passes ... passed
+        """
+
+
+    @with_configfile
+    Scenario: Can override default formatter from configfile on command-line
+      Given a file named "behave.ini" with:
+        """
+        [behave]
+        default_format = plain
+        show_timings = false
+        """
+      When I run "behave -f progress features/"
+      Then it should pass with:
+        """
+        2 features passed, 0 failed, 0 skipped
+        2 scenarios passed, 0 failed, 0 skipped
+        """
+      And the command output should contain:
+        """
+        features/alice.feature  .
+        features/bob.feature  .
+        """
+
+
+    # -- Some formatter are specified in configfile.
+    @with_configfile
+    Scenario: Default formatter is used when non is provided on command-line
+      Given a file named "behave.ini" with:
+        """
+        [behave]
+        default_format = plain
+        format = progress
+        outfiles = output/progress.out
+        show_timings = false
+        """
+      And I remove the directory "output"
+      When I run "behave features/"
+      Then it should pass with:
+        """
+        2 features passed, 0 failed, 0 skipped
+        2 scenarios passed, 0 failed, 0 skipped
+        """
+      And the command output should contain:
+        """
+        Feature: Alice
+          Scenario: A1
+            Given a step passes ... passed
+            When a step passes ... passed
+            Then a step passes ... passed
+
+        Feature: Bob
+          Scenario: B1
+            When a step passes ... passed
+            Then a step passes ... passed
+        """
+      And the file "output/progress.out" should contain:
+        """
+        features/alice.feature  .
+        features/bob.feature  .
+        """