busser-behave 0.1.3

data/vendor/behave/features/logcapture.feature

@@ -0,0 +1,188 @@
+@logging
+@capture
+Feature: Capture log output
+
+  As a tester
+  I want that log output is captured
+  But log-records are only shown when failures/errors occur
+  So that failure diagnostics are simplified
+
+  . SPECIFICATION:
+  .  * log_capture mode is enabled per default
+  .  * log_capture mode can be defined on command-line
+  .  * log_capture mode can be defined in behave configuration file
+  .  * In log_capture mode: Captured log-records are only shown if a scenario fails
+  .
+  . RELATED:
+  .  * logcapture.*.feature
+  .  * logging.*.feature
+
+  @setup
+  Scenario: Feature Setup
+    Given a new working directory
+    And a file named "features/steps/use_behave4cmd_steps.py" with:
+        """
+        import behave4cmd0.log.steps
+        import behave4cmd0.failing_steps
+        import behave4cmd0.passing_steps
+        """
+    And a file named "features/environment.py" with:
+        """
+        def before_all(context):
+            context.config.setup_logging()
+            # -- SAME-AS:
+            # import logging
+            # logging.basicConfig(level=context.config.logging_level)
+        """
+    And  a file named "features/example.log_and_pass.feature" with:
+        """
+        Feature:
+          Scenario: Passing
+            Given I create log records with:
+                | category | level   | message |
+                | root     |  FATAL  | Hello Alice  |
+                | foo      |  ERROR  | Hello Bob    |
+                | foo.bar  |  WARN   | Hello Charly |
+                | bar      |  INFO   | Hello Dora   |
+                | baz      |  DEBUG  | Hello Emily  |
+            When another step passes
+        """
+    And  a file named "features/example.log_and_fail.feature" with:
+        """
+        Feature:
+          Scenario: Failing
+            Given I create log records with:
+                | category | level   | message |
+                | root     |  FATAL  | Hello Alice  |
+                | foo      |  ERROR  | Hello Bob    |
+                | foo.bar  |  WARN   | Hello Charly |
+                | bar      |  INFO   | Hello Dora   |
+                | baz      |  DEBUG  | Hello Emily  |
+            When another step fails
+        """
+
+
+    Scenario: Captured log is suppressed if scenario passes
+        When I run "behave -f plain -T --logcapture features/example.log_and_pass.feature"
+        Then it should pass with:
+            """
+            1 scenario passed, 0 failed, 0 skipped
+            2 steps passed, 0 failed, 0 skipped, 0 undefined
+            """
+        And the command output should contain:
+            """
+            Scenario: Passing
+              Given I create log records with ... passed
+                | category | level | message      |
+                | root     | FATAL | Hello Alice  |
+                | foo      | ERROR | Hello Bob    |
+                | foo.bar  | WARN  | Hello Charly |
+                | bar      | INFO  | Hello Dora   |
+                | baz      | DEBUG | Hello Emily  |
+              When another step passes ... passed
+            """
+        And the command output should not contain:
+            """
+            Captured logging:
+            """
+        But the command output should not contain the following log records:
+            | category | level   | message |
+            | root     |  FATAL  | Hello Alice  |
+            | foo      |  ERROR  | Hello Bob    |
+            | foo.bar  |  WARN   | Hello Charly |
+            | bar      |  INFO   | Hello Dora   |
+            | baz      |  DEBUG  | Hello Emily  |
+
+
+    Scenario: Captured log is shown up to first failure if scenario fails
+        When I run "behave -f plain -T --logcapture features/example.log_and_fail.feature"
+        Then it should fail with:
+            """
+            0 scenarios passed, 1 failed, 0 skipped
+            1 step passed, 1 failed, 0 skipped, 0 undefined
+            """
+        And the command output should contain:
+            """
+            Feature:
+              Scenario: Failing
+                Given I create log records with ... passed
+                  | category | level | message      |
+                  | root     | FATAL | Hello Alice  |
+                  | foo      | ERROR | Hello Bob    |
+                  | foo.bar  | WARN  | Hello Charly |
+                  | bar      | INFO  | Hello Dora   |
+                  | baz      | DEBUG | Hello Emily  |
+                When another step fails ... failed
+            Assertion Failed: EXPECT: Failing step
+            Captured logging:
+            CRITICAL:root:Hello Alice
+            ERROR:foo:Hello Bob
+            WARNING:foo.bar:Hello Charly
+            INFO:bar:Hello Dora
+            """
+        And the command output should contain the following log records:
+            | category | level   | message |
+            | root     |  FATAL  | Hello Alice  |
+            | foo      |  ERROR  | Hello Bob    |
+            | foo.bar  |  WARN   | Hello Charly |
+            | bar      |  INFO   | Hello Dora   |
+        But the command output should not contain the following log records:
+            | category | level   | message     | Comment |
+            | baz      |  DEBUG  | Hello Emily | Log-level too low: filtered-out |
+
+
+    Scenario: Captured log is shown up to first failure if log_record.level is not too low
+
+      Ensure that only log-records are shown that exceed the logging-level.
+
+        When I run "behave -f plain --logcapture --logging-level=ERROR features/example.log_and_fail.feature"
+        Then it should fail
+        And the command output should contain the following log records:
+            | category | level   | message |
+            | root     |  FATAL  | Hello Alice  |
+            | foo      |  ERROR  | Hello Bob    |
+        But the command output should not contain the following log records:
+            | category | level   | message |
+            | foo.bar  |  WARN   | Hello Charly |
+            | bar      |  INFO   | Hello Dora   |
+            | baz      |  DEBUG  | Hello Emily  |
+
+
+    Scenario: Logcapture mode is enabled per default
+      When I run "behave -f plain features/example.log_and_fail.feature"
+      Then it should fail
+      And the command output should contain "Captured logging:"
+
+
+    Scenario: Logcapture mode can be enabled on command-line
+      When I run "behave -f plain --logcapture features/example.log_and_fail.feature"
+      Then it should fail
+      And the command output should contain "Captured logging:"
+
+
+    Scenario: Logcapture mode can be disabled on command-line
+      When I run "behave -f plain --no-logcapture features/example.log_and_fail.feature"
+      Then it should fail
+      And the command output should not contain "Captured logging:"
+
+
+    Scenario: Logcapture mode can be enabled in configfile
+      Given a file named "behave.ini" with:
+          """
+          [behave]
+          log_capture = true
+          """
+      When I run "behave -f plain features/example.log_and_fail.feature"
+      Then it should fail
+      And the command output should contain "Captured logging:"
+
+
+    Scenario: Logcapture mode can be disabled in configfile
+      Given a file named "behave.ini" with:
+          """
+          [behave]
+          log_capture = false
+          """
+      When I run "behave -f plain features/example.log_and_fail.feature"
+      Then it should fail
+      And the command output should not contain "Captured logging:"

data/vendor/behave/features/logcapture.filter.feature

@@ -0,0 +1,130 @@
+Feature: Use logging_filter with logcapture
+
+  PRECONDITION: log_capture mode is enabled (config.log_capture = true).
+
+  As a tester
+  In log-capture mode
+  I want to include/exclude log-records from some logging categories
+  So that the output is not cluttered with unneeded information in case of failures.
+
+    Background:
+      Given I define the log record schema:
+          | category | level | message         |
+          | root     | ERROR | __LOG_MESSAGE__ |
+
+    @setup
+    Scenario: Feature Setup
+      Given a new working directory
+      And a file named "features/steps/use_behave4cmd_steps.py" with:
+          """
+          import behave4cmd0.log.steps
+          import behave4cmd0.failing_steps
+          import behave4cmd0.passing_steps
+          """
+      And a file named "features/logging.failing_example.feature" with:
+          """
+          Feature:
+            Scenario: Failing
+              Given I create log records with:
+                  | category | level   | message |
+                  | root     |  ERROR  | __LOG_MESSAGE__ |
+                  | foo      |  ERROR  | __LOG_MESSAGE__ |
+                  | foo.bar  |  ERROR  | __LOG_MESSAGE__ |
+                  | bar      |  ERROR  | __LOG_MESSAGE__ |
+              When another step fails
+          """
+      And a file named "behave.ini" with:
+          """
+          [behave]
+          log_capture = true
+          logging_level = WARN
+          """
+
+
+    Scenario: Include only a logging category
+      When I run "behave --logcapture --logging-filter=foo features/logging.failing_example.feature"
+      Then it should fail with:
+          """
+          0 scenarios passed, 1 failed, 0 skipped
+          1 step passed, 1 failed, 0 skipped, 0 undefined
+          """
+      And the command output should contain log records from categories:
+          | category |
+          | foo      |
+      But the command output should not contain log records from categories:
+          | category | Comment |
+          | root     | Not included: filtered-out |
+          | foo.bar  | Not included: filtered-out |
+          | bar      | Not included: filtered-out |
+
+
+    Scenario: Include only a logging sub-category
+      When I run "behave --logcapture --logging-filter=foo.bar features/logging.failing_example.feature"
+      Then it should fail
+      And the command output should contain log records from categories:
+          | category | Comment  |
+          | foo.bar  | Included |
+      But the command output should not contain log records from categories:
+          | category | Comment |
+          | root     | Not included: filtered-out |
+          | foo      | Not included: filtered-out |
+          | bar      | Not included: filtered-out |
+
+
+    Scenario: Exclude a logging category
+      When I run "behave --logcapture --logging-filter=-foo features/logging.failing_example.feature"
+      Then it should fail
+      And the command output should contain log records from categories:
+          | category | Comment |
+          | root     | Not excluded: foo |
+          | foo.bar  | Not excluded: foo |
+          | bar      | Not excluded: foo |
+      But the command output should not contain log records from categories:
+          | category | Comment |
+          | foo      | Excluded |
+
+
+    Scenario: Include several logging categories
+      When I run "behave --logcapture --logging-filter=foo,bar features/logging.failing_example.feature"
+      Then it should fail
+      And the command output should contain log records from categories:
+          | category | Comment |
+          | foo      | Included: foo |
+          | bar      | Included: bar |
+      But the command output should not contain log records from categories:
+          | category | Comment |
+          | root     | Not included: filtered-out  |
+          | foo.bar  | Not included (sub-category) |
+
+
+    Scenario: Include/exclude several logging categories
+      When I run "behave --logcapture --logging-filter=foo.bar,-bar features/logging.failing_example.feature"
+      Then it should fail
+      And the command output should contain log records from categories:
+          | category | Comment |
+          | root     | Not excluded: bar |
+          | foo      | Not excluded: bar |
+          | foo.bar  | Included  |
+      But the command output should not contain log records from categories:
+          | category | Comment |
+          | bar      | Excluded: filtered-out |
+
+
+    Scenario: Include/exclude several logging categories with configfile
+      Given a file named "behave.ini" with:
+          """
+          [behave]
+          log_capture = true
+          logging_level = WARN
+          logging_filter = foo.bar,-bar
+          """
+      When I run "behave --logcapture features/logging.failing_example.feature"
+      Then it should fail
+      And the command output should contain log records from categories:
+          | category | Comment |
+          | root     | Not excluded: bar |
+          | foo      | Not excluded: bar |
+          | foo.bar  | Included  |
+      But the command output should not contain log records from categories:
+          | category | Comment |
+          | bar      | Excluded: filtered-out |

data/vendor/behave/features/logging.no_capture.feature

@@ -0,0 +1,99 @@
+@logging
+@no_capture
+Feature: No-logcapture mode (normal mode) shows log-records
+
+  As a tester
+  I want that sometimes that log output is not captured
+  So that I can see progress even in case of success (or failures)
+
+
+  @setup
+  Scenario: Feature Setup
+    Given a new working directory
+    And a file named "features/steps/use_behave4cmd_steps.py" with:
+        """
+        import behave4cmd0.log.steps
+        import behave4cmd0.failing_steps
+        import behave4cmd0.passing_steps
+        """
+
+
+    Scenario: Log records are shown when scenario passes (CASE 1)
+
+      Ensure that log-records are filtered out if log_record.level is too low.
+
+        Given a file named "features/environment.py" with:
+            """
+            def before_all(context):
+                import logging
+                context.config.setup_logging(logging.WARN)
+                # -- SAME AS:
+                # logging.basicConfig()
+                # logging.getLogger().setLevel(logging.WARN)
+            """
+        And a file named "features/logging.passing_example.feature" with:
+            """
+            Feature:
+              Scenario: Passing
+                Given I create log records with:
+                    | category | level   | message |
+                    | root     |  ERROR  | Hello Alice  |
+                    | root     |  WARN   | Hello Bob    |
+                    | root     |  INFO   | Hello Charly |
+                    | root     |  DEBUG  | Hello Doro   |
+                When another step passes
+            """
+        When I run "behave -f plain -T --no-logcapture features/logging.passing_example.feature"
+        Then it should pass
+        And the command output should contain the following log records:
+            | category | level   | message |
+            | root     |  ERROR  | Hello Alice  |
+            | root     |  WARN   | Hello Bob    |
+        But the command output should not contain the following log records:
+            | category | level   | message      | Comment |
+            | root     |  INFO   | Hello Charly | Filtered-out |
+            | root     |  DEBUG  | Hello Doro   | Filtered-out |
+        And note that "log_records with level below WARN are filtered out"
+
+
+    @no_capture
+    Scenario: Log records are shown up to first failing step (CASE 2)
+
+      Ensure that log-records are filtered out if log_record.level is too low.
+
+        Given a file named "features/environment.py" with:
+            """
+            def before_all(context):
+                context.config.setup_logging()
+            """
+        And a file named "features/logging.failing_example.feature" with:
+            """
+            Feature:
+              Scenario: Failing
+                Given I create log records with:
+                    | category | level   | message |
+                    | root     |  ERROR  | Hello Alice  |
+                    | root     |  WARN   | Hello Bob    |
+                    | root     |  INFO   | Hello Charly |
+                    | root     |  DEBUG  | Hello Doro   |
+                When another step fails
+                Then I create log records with:
+                    | category | level   | message |
+                    | root     |  ERROR  | Hello2 Zerberus |
+            """
+        When I run "behave -f plain -T --no-logcapture --logging-level=ERROR features/logging.failing_example.feature"
+        Then it should fail with:
+            """
+            0 scenarios passed, 1 failed, 0 skipped
+            1 step passed, 1 failed, 1 skipped, 0 undefined
+            """
+        And the command output should contain the following log records:
+            | category | level   | message |
+            | root     |  ERROR  | Hello Alice  |
+        But the command output should not contain the following log records:
+            | category | level   | message         | Comment |
+            | root     |  WARN   | Hello Bob       | Filtered-out |
+            | root     |  INFO   | Hello Charly    | Filtered-out |
+            | root     |  DEBUG  | Hello Doro      | Filtered-out |
+            | root     |  ERROR  | Hello2 Zerberus | Skipped      |
+        And note that "log_records with level below ERROR are filtered out"

data/vendor/behave/features/logging.setup_format.feature

@@ -0,0 +1,157 @@
+Feature: Setup logging_format
+
+  As a tester
+  I want to configure the logging_format for log_capture mode
+  So that log-records are shown in my preferred format.
+
+  As a tester
+  I want to configure the logging_format for logging mode (no-log_capture)
+  So that log-records are shown in my preferred format.
+  
+  . SPECIFICATION:
+  .  * logging_format can be defined on command-line
+  .  * logging_format can be defined in behave configuration file
+  .
+  . NOTE:
+  .  The log record format can also be defined in a logging configuration file.
+
+  @setup
+  Scenario: Feature Setup
+    Given a new working directory
+    And a file named "features/steps/use_behave4cmd_steps.py" with:
+        """
+        import behave4cmd0.log.steps
+        import behave4cmd0.failing_steps
+        import behave4cmd0.passing_steps
+        """
+    And a file named "features/environment.py" with:
+        """
+        def before_all(context):
+            context.config.setup_logging()
+        """
+    And  a file named "features/example.log_and_fail.feature" with:
+        """
+        Feature:
+          Scenario: Failing
+            Given I create log records with:
+                | category | level   | message |
+                | root     |  ERROR  | Hello Alice |
+                | root     |  WARN   | Hello Bob   |
+            When a step fails
+        """
+
+  @capture
+  Scenario: Use logging_format on command-line (case: log_capture mode)
+    Given a file named "behave.ini" with:
+        """
+        [behave]
+        log_capture = true
+        logging_level = WARN
+        """
+    When I run "behave -f plain -T --logging-format='LOG.%(levelname)-8s  %(name)-10s: %(message)s' features/"
+    Then it should fail with:
+        """
+        0 scenarios passed, 1 failed, 0 skipped
+        1 step passed, 1 failed, 0 skipped, 0 undefined
+        """
+    And the command output should contain "Captured logging:"
+    And the command output should contain:
+        """
+        Captured logging:
+        LOG.ERROR     root      : Hello Alice
+        LOG.WARNING   root      : Hello Bob
+        """
+    When I use the log record configuration:
+        | property | value |
+        | format   | LOG.%(levelname)-8s  %(name)-10s: %(message)s |
+    Then the command output should contain the following log records:
+        | category | level   | message |
+        | root     |  ERROR  | Hello Alice |
+        | root     |  WARN   | Hello Bob   |
+
+
+  @capture
+  Scenario: Use logging_format in config-file (case: log_capture mode)
+    Given a file named "behave.ini" with:
+        """
+        [behave]
+        log_capture = true
+        logging_level = WARN
+        logging_format = LOG.%(levelname)-8s  %(name)-10s: %(message)s
+        """
+    When I run "behave -f plain features/"
+    Then it should fail
+    And the command output should contain "Captured logging:"
+    And the command output should contain:
+        """
+        Captured logging:
+        LOG.ERROR     root      : Hello Alice
+        LOG.WARNING   root      : Hello Bob
+        """
+
+  @no_capture
+  Scenario: Use logging_format on command-line (case: logging mode)
+    Given a file named "behave.ini" with:
+        """
+        [behave]
+        log_capture = false
+        logging_level = WARN
+        """
+    When I run "behave -f plain -T --logging-format='LOG.%(levelname)-8s  %(name)-10s: %(message)s' features/"
+    Then it should fail with:
+        """
+        0 scenarios passed, 1 failed, 0 skipped
+        1 step passed, 1 failed, 0 skipped, 0 undefined
+        """
+    And the command output should not contain "Captured logging:"
+    And the command output should contain:
+        """
+        LOG.ERROR     root      : Hello Alice
+        LOG.WARNING   root      : Hello Bob
+        """
+    When I use the log record configuration:
+        | property | value |
+        | format   | LOG.%(levelname)-8s  %(name)-10s: %(message)s |
+    Then the command output should contain the following log records:
+        | category | level   | message |
+        | root     |  ERROR  | Hello Alice |
+        | root     |  WARN   | Hello Bob   |
+
+
+  @no_capture
+  Scenario: Use logging_format in config-file (case: logging mode)
+    Given a file named "behave.ini" with:
+        """
+        [behave]
+        log_capture = false
+        logging_level = WARN
+        logging_format = LOG.%(levelname)-8s  %(name)-10s: %(message)s
+        """
+    When I run "behave -f plain features/"
+    Then it should fail
+    And the command output should not contain "Captured logging:"
+    And the command output should contain:
+        """
+        LOG.ERROR     root      : Hello Alice
+        LOG.WARNING   root      : Hello Bob
+        """
+
+  @capture
+  Scenario: Use logging_datefmt in config-file
+
+    Ensure that "logging_datefmt" option can be used.
+
+    Given a file named "behave.ini" with:
+        """
+        [behave]
+        logging_format = %(asctime)s  LOG.%(levelname)-8s %(name)s: %(message)s
+        logging_datefmt = %Y-%m-%dT%H:%M:%S
+        """
+    When I run "behave -f plain features/"
+    Then it should fail with:
+        """
+        0 scenarios passed, 1 failed, 0 skipped
+        1 step passed, 1 failed, 0 skipped, 0 undefined
+        """
+    And the command output should contain "LOG.ERROR    root: Hello Alice"
+    And the command output should contain "LOG.WARNING  root: Hello Bob"