busser-behave 0.1.3

data/vendor/behave/features/runner.multiple_formatters.feature

@@ -0,0 +1,285 @@
+Feature: Multiple Formatter with different outputs
+
+  . SPECIFICATION: Command-line option --format
+  .  * Each --format option specifies one formatter to use.
+  .
+  . SPECIFICATION: Command-line option --outfile
+  .  * Multiple --outfile options can be provided
+  .  * The nth --outfile option is used for the nth formatter
+  .  * If less --outfile options are provided than formatter,
+  .    the remaining formatter use stdout as output stream.
+  .    Therefore, the last formatter should in general use stdout.
+  .
+  . SPECIFICATION: Configuration file
+  .  * Option format with one or more formatters can be used (optional).
+  .  * Formatters specified in the configuration file are always executed.
+  .  * If not enough outfiles are specified, the outfiles list is extended
+  .    by using an outfile "${format}.output" for each missing outfile.
+  .
+  . RELATED TO:
+  .  * issue #47  Formatter chain is broken
+
+
+    @setup
+    Scenario: Test Setup
+      Given a new working directory
+      And a file named "features/steps/passing_steps.py" with:
+        """
+        from behave import step
+
+        @step('a step passes')
+        def step_passes(context):
+            pass
+        """
+      And a file named "features/alice.feature" with:
+        """
+        Feature: Alice
+          Scenario: A1
+            Given a step passes
+            When a step passes
+            Then a step passes
+        """
+      And a file named "features/bob.feature" with:
+        """
+        Feature: Bob
+          Scenario: B1
+            When a step passes
+            Then a step passes
+        """
+
+    @no_configfile
+    Scenario: One formatter, no outfile (use stdout instead)
+      Given a file named "behave.ini" does not exist
+      When I run "behave -f plain -T features/"
+      Then it should pass with:
+        """
+        2 features passed, 0 failed, 0 skipped
+        2 scenarios passed, 0 failed, 0 skipped
+        """
+      And the command output should contain:
+        """
+        Feature: Alice
+          Scenario: A1
+            Given a step passes ... passed
+            When a step passes ... passed
+            Then a step passes ... passed
+
+        Feature: Bob
+          Scenario: B1
+            When a step passes ... passed
+            Then a step passes ... passed
+        """
+
+    @no_configfile
+    Scenario: One formatter, one outfile
+      Given a file named "behave.ini" does not exist
+      When I run "behave -f plain --outfile=output/plain.out -T features/"
+      Then it should pass with:
+        """
+        2 features passed, 0 failed, 0 skipped
+        2 scenarios passed, 0 failed, 0 skipped
+        """
+      And the command output should not contain:
+        """
+        Feature: Alice
+          Scenario: A1
+        """
+      But a file named "output/plain.out" should exist
+      And the file "output/plain.out" should contain:
+        """
+        Feature: Alice
+          Scenario: A1
+            Given a step passes ... passed
+            When a step passes ... passed
+            Then a step passes ... passed
+
+        Feature: Bob
+          Scenario: B1
+            When a step passes ... passed
+            Then a step passes ... passed
+        """
+
+    @no_configfile
+    Scenario: Two formatter, one outfile
+      Given a file named "behave.ini" does not exist
+      When I run "behave -f plain -o output/plain.out -f progress -T features/"
+      Then it should pass with:
+        """
+        2 features passed, 0 failed, 0 skipped
+        2 scenarios passed, 0 failed, 0 skipped
+        """
+      And the command output should contain:
+        """
+        features/alice.feature  .
+        features/bob.feature  .
+        """
+      But a file named "output/plain.out" should exist
+      And the file "output/plain.out" should contain:
+        """
+        Feature: Alice
+          Scenario: A1
+            Given a step passes ... passed
+            When a step passes ... passed
+            Then a step passes ... passed
+
+        Feature: Bob
+          Scenario: B1
+            When a step passes ... passed
+            Then a step passes ... passed
+        """
+
+
+    @no_configfile
+    Scenario: More outfiles than formatter should fail with CONFIG-ERROR
+      Given a file named "behave.ini" does not exist
+      When I run "behave -f plain -o plain.output -o xxx.output features/"
+      Then it should fail with:
+        """
+        CONFIG-ERROR: More outfiles (2) than formatters (1).
+        """
+
+    @with_configfile
+    Scenario: Use default formatter and outfile from behave configuration file
+      Given a file named "behave.ini" with:
+        """
+        [behave]
+        format   = plain
+        outfiles = output/plain.out
+        show_timings = false
+        """
+      And I remove the directory "output"
+      When I run "behave features/"
+      Then it should pass
+      And the command output should not contain:
+        """
+        Feature: Alice
+          Scenario: A1
+        """
+      But a file named "output/plain.out" should exist
+      And the file "output/plain.out" should contain:
+        """
+        Feature: Alice
+          Scenario: A1
+            Given a step passes ... passed
+            When a step passes ... passed
+            Then a step passes ... passed
+
+        Feature: Bob
+          Scenario: B1
+            When a step passes ... passed
+            Then a step passes ... passed
+        """
+
+    @with_configfile
+    Scenario: Use default formatter and another without outfile from behave configuration file
+      Given a file named "behave.ini" with:
+        """
+        [behave]
+        default_format = plain
+        format = progress
+        # -- OOPS: No outfile specified => Use "${format}.output" as outfile
+        show_timings = false
+        """
+      And I remove the directory "output"
+      When I run "behave features/"
+      Then it should pass
+      And the command output should contain:
+        """
+        Feature: Alice
+          Scenario: A1
+            Given a step passes ... passed
+            When a step passes ... passed
+            Then a step passes ... passed
+
+        Feature: Bob
+          Scenario: B1
+            When a step passes ... passed
+            Then a step passes ... passed
+        """
+      And a file named "progress.output" should exist
+      And the file "progress.output" should contain:
+        """
+        features/alice.feature  .
+        features/bob.feature  .
+        """
+
+
+    @with_configfile
+    Scenario: Command-line formatter/outfile extend behave configuration file args
+      Given a file named "behave.ini" with:
+        """
+        [behave]
+        show_timings = false
+        format   = plain
+        outfiles = output/plain.out
+        """
+      And I remove the directory "output"
+      When I run "behave -c -f pretty -o output/pretty.out -f progress -o output/progress.out features/"
+      Then it should pass
+      And the file "output/progress.out" should contain:
+        """
+        features/alice.feature  .
+        features/bob.feature  .
+        """
+      And the file "output/pretty.out" should contain:
+        """
+        Feature: Alice # features/alice.feature:1
+          Scenario: A1          # features/alice.feature:2
+            Given a step passes # features/steps/passing_steps.py:3
+            When a step passes  # features/steps/passing_steps.py:3
+            Then a step passes  # features/steps/passing_steps.py:3
+
+        Feature: Bob # features/bob.feature:1
+          Scenario: B1         # features/bob.feature:2
+            When a step passes # features/steps/passing_steps.py:3
+            Then a step passes # features/steps/passing_steps.py:3
+        """
+      And a file named "output/plain.out" should exist
+      And the file "output/plain.out" should contain:
+        """
+        Feature: Alice
+          Scenario: A1
+            Given a step passes ... passed
+            When a step passes ... passed
+            Then a step passes ... passed
+
+        Feature: Bob
+          Scenario: B1
+            When a step passes ... passed
+            Then a step passes ... passed
+        """
+
+
+    @with_configfile
+    Scenario: Combination of formatter from configfile and command-line cannot cause outfile offsets
+      Given a file named "behave.ini" with:
+        """
+        [behave]
+        show_timings = false
+        format = plain
+        # -- OOPS: No outfiles defined => Use "${format}.output" as outfile.
+        """
+      And I remove the directory "output"
+      When I run "behave -f progress -o output/progress.out features/"
+      Then it should pass
+      And the command output should not contain:
+        """
+        Feature: Alice
+          Scenario: A1
+            Given a step passes ... passed
+            When a step passes ... passed
+            Then a step passes ... passed
+        """
+      But the file "plain.output" should contain:
+        """
+        Feature: Alice
+          Scenario: A1
+            Given a step passes ... passed
+            When a step passes ... passed
+            Then a step passes ... passed
+        """
+      And the file "output/progress.out" should contain:
+        """
+        features/alice.feature  .
+        features/bob.feature  .
+        """

data/vendor/behave/features/runner.scenario_autoretry.feature

@@ -0,0 +1,131 @@
+Feature: Auto-retry failed scenarios (a number of times)
+
+  As a tester in an unreliable environment
+  I want that failed scenarios are automatically retried a number of times
+  So that these scenarios are passing
+
+  NOTE: 
+  This problem should occur rather seldom, but may occur when the server or 
+  network infrastructure, etc. is unreliable and causes random failures.
+
+
+  @setup
+  Scenario: Feature Setup
+    Given a new working directory
+    And a file named "features/steps/unreliable_steps.py" with:
+        """
+        from behave import given, step
+
+        unreliable_step_passed_calls = 0
+
+        @step(u'{word:w} unreliable step fails sometimes')
+        def step_unreliable_step_fails_sometimes(context, word):
+            global unreliable_step_passed_calls
+            userdata = context.config.userdata
+            fault_pos = userdata.getint("UNRELIABLE_FAULTPOS", 0)
+            if fault_pos <= 0:
+                return
+
+            # -- NORMAL CASE: Unreliable step should every N calls.
+            unreliable_step_passed_calls += 1
+            if unreliable_step_passed_calls >= fault_pos:
+                unreliable_step_passed_calls = 0
+                assert False, "UNRELIABLE-STEP FAILURE"
+        """
+    And a file named "features/steps/reuse_steps.py" with:
+        """
+        import behave4cmd0.passing_steps
+        """
+    And a file named "features/environment.py" with:
+        """
+        from behave.contrib.scenario_autoretry import patch_scenario_with_autoretry
+
+        def before_feature(context, feature):
+            for scenario in feature.scenarios:
+                if "autoretry" in scenario.effective_tags:
+                    patch_scenario_with_autoretry(scenario, max_attempts=2)
+        """
+    And a file named "features/unreliable.feature" with:
+        """
+        @autoretry
+        Feature: Alice
+            Scenario: A1
+              Given a step passes
+              When an unreliable step fails sometimes
+              Then another unreliable step fails sometimes
+
+            Scenario: A2
+              Given an unreliable step fails sometimes
+              Then another step passes  
+        """
+    And a file named "behave.ini" with:
+        """
+        [behave]
+        show_timings = false
+        """
+
+
+  Scenario: Retry failed scenarios
+    When I run "behave -f plain -D UNRELIABLE_FAULTPOS=3 features/unreliable.feature"
+    Then it should pass with:
+        """
+        2 scenarios passed, 0 failed, 0 skipped
+        5 steps passed, 0 failed, 0 skipped, 0 undefined
+        """
+    And the command output should contain:
+        """
+        AUTO-RETRY SCENARIO (attempt 1)
+        """
+    But the command output should not contain:
+        """
+        AUTO-RETRY SCENARIO (attempt 2)
+        """
+    And the command output should contain:
+        """
+        Scenario: A1
+          Given a step passes ... passed
+          When an unreliable step fails sometimes ... passed
+          Then another unreliable step fails sometimes ... passed
+      
+        Scenario: A2
+          Given an unreliable step fails sometimes ... failed
+        Assertion Failed: UNRELIABLE-STEP FAILURE
+        AUTO-RETRY SCENARIO (attempt 1)
+      
+        Scenario: A2
+          Given an unreliable step fails sometimes ... passed
+          Then another step passes ... passed
+        """
+
+  Scenario: Scenarios failures are accepted after N=2 attempts
+    When I run "behave -f plain -D UNRELIABLE_FAULTPOS=2 features/unreliable.feature"
+    Then it should fail with:
+        """
+        1 scenario passed, 1 failed, 0 skipped
+        4 steps passed, 1 failed, 0 skipped, 0 undefined
+        """
+    And the command output should not contain:
+        """
+        AUTO-RETRY SCENARIO (attempt 2)
+        SCENARIO-FAILURE ACCEPTED (after 2 attempts)
+        """
+    And the command output should contain:
+        """
+        Scenario: A1
+          Given a step passes ... passed
+          When an unreliable step fails sometimes ... passed
+          Then another unreliable step fails sometimes ... failed
+        Assertion Failed: UNRELIABLE-STEP FAILURE
+        AUTO-RETRY SCENARIO (attempt 1)
+        
+        Scenario: A1
+          Given a step passes ... passed
+           When an unreliable step fails sometimes ... passed
+          Then another unreliable step fails sometimes ... failed
+        Assertion Failed: UNRELIABLE-STEP FAILURE
+        AUTO-RETRY SCENARIO FAILED (after 2 attempts)
+      
+        Scenario: A2
+          Given an unreliable step fails sometimes ... passed
+          Then another step passes ... passed
+        """        

data/vendor/behave/features/runner.select_files_by_regexp.example.feature

@@ -0,0 +1,71 @@
+Feature: Select feature files by using regular expressions (self-test)
+
+  Use behave self-tests to ensure that --incude/--exclude options work.
+  RELATED: runner.select_files_by_regexp.feature
+
+  @setup
+  Scenario: Feature Setup
+    Given a new working directory
+    And an empty file named "features/steps/steps.py"
+    And a file named "features/alice.feature" with:
+        """
+        Feature: Alice
+          Scenario: A1
+        """
+    And a file named "features/barbi.feature" with:
+        """
+        Feature: Barbi
+          Scenario: B1
+        """
+    And a file named "features/bob.feature" with:
+        """
+        Feature: Bob
+          Scenario: B2
+        """
+
+
+  Scenario: Include only feature files
+
+    Select the following feature files: barbi.feature, bob.feature
+
+      When I run "behave --include='features/b.*' -f plain features/"
+      Then it should pass with:
+        """
+        2 features passed, 0 failed, 0 skipped
+        """
+      And the command output should contain:
+        """
+        Feature: Barbi
+          Scenario: B1
+
+        Feature: Bob
+          Scenario: B2
+        """
+
+  Scenario: Exclude only feature files
+
+    Select the following feature files: alice.feature
+
+      When I run "behave --exclude='features/b.*' -f plain features/"
+      Then it should pass with:
+        """
+        1 feature passed, 0 failed, 0 skipped
+        """
+      And the command output should contain:
+        """
+        Feature: Alice
+        """
+
+  Scenario: Include and exclude feature files
+
+    Select the following feature files: alice.feature
+
+      When I run "behave --include='features/.*a.*\.feature' --exclude='.*/barbi.*' -f plain features/"
+      Then it should pass with:
+        """
+        1 feature passed, 0 failed, 0 skipped
+        """
+      And the command output should contain:
+        """
+        Feature: Alice
+        """