cuke_cataloger 1.0.0

data/features/tag_indexing.feature

@@ -0,0 +1,362 @@
+Feature: Tag indexing
+
+  The tool is capable of determining which indexes are already in use in a test suite. If no other set of indexes is
+  provided, these existing indexes are what will be used to pick what indexes the tool will start with when tagging.
+  Using this default, new indexes start after the highest found index so as not to repeat the indexes of older tests
+  that might have since been removed. This default behavior can be overridden by providing an initial index from which
+  to start tagging.
+
+  An analogous principle is used for outlines.
+
+
+  Background:
+    * a tag prefix of "@test_case_"
+
+  Scenario: Used indexes can be determined
+    Given the following feature file "file1.feature":
+    """
+    Feature:
+
+      Scenario:
+        * a step
+
+      @test_case_1
+      Scenario:
+        * a step
+
+      @test_case_7
+      Scenario Outline:
+        * a step
+      Examples: with rows
+        | test_case_id | param 1 |
+        | 7-1          | value 1 |
+      Examples: without rows
+        | test_case_id | param 1 |
+      Examples: with empty rows
+        | test_case_id | param 1 |
+        |              | value 1 |
+        | 7-2          | value 2 |
+      Examples: without an id parameter
+        | param 1 |
+        | value 1 |
+
+      #Missing the parent tag but the sub-tags still count
+      Scenario Outline:
+        * a step
+      Examples: with rows
+        | test_case_id | param 1 |
+        | 8-1          | value 1 |
+      Examples: with a non-indexed id
+        | test_case_id | param 1 |
+        | trash        | value 1 |
+    """
+    And the following feature file "file2.feature":
+    """
+    Feature: Just another feature to make sure that the entire suite is checked
+
+      @test_case_5
+      Scenario:
+        * a step
+    """
+    When the existing ids are determined
+    Then the following ids are found:
+      | 1   |
+      | 5   |
+      | 7   |
+      | 7-1 |
+      | 7-2 |
+      | 8-1 |
+
+
+  Scenario: No existing tags
+    Given the following feature file:
+    """
+    Feature:
+
+      Scenario:
+        * a step
+    """
+    When the file is processed
+    Then the resulting file is:
+    """
+    Feature:
+
+      @test_case_1
+      Scenario:
+        * a step
+    """
+
+  Scenario: Existing tags
+    Given the following feature file:
+    """
+    Feature:
+
+      Scenario:
+        * a step
+
+      @test_case_1
+      Scenario:
+        * a step
+    """
+    And the following feature file:
+    """
+    Feature: Another file to test that the entire suite is checked
+
+      @test_case_5
+      Scenario:
+        * a step
+    """
+    When the file is processed
+    Then the resulting file "1" is:
+    """
+    Feature:
+
+      @test_case_6
+      Scenario:
+        * a step
+
+      @test_case_1
+      Scenario:
+        * a step
+    """
+    And the resulting file "2" is:
+    """
+    Feature: Another file to test that the entire suite is checked
+
+      @test_case_5
+      Scenario:
+        * a step
+    """
+
+  Scenario: Explicit starting index
+
+  Note: By explicitly setting the starting index, there is no guarantee that the indexes used will not conflict with
+  existing indexes.
+
+    Given the following feature file:
+    """
+    Feature:
+
+      Scenario:
+        * a step
+
+      @test_case_1
+      Scenario:
+        * a step
+
+      Scenario:
+        * a step
+    """
+    And a start index of "1"
+    When the file is processed
+    Then the resulting file is:
+    """
+    Feature:
+
+      @test_case_1
+      Scenario:
+        * a step
+
+      @test_case_1
+      Scenario:
+        * a step
+
+      @test_case_2
+      Scenario:
+        * a step
+    """
+
+
+  Scenario: Outline, no existing sub-ids
+    Given the following feature file:
+    """
+    Feature:
+
+      Scenario Outline:
+        * a <param>
+      Examples:
+        | param |
+        | value |
+    """
+    When the file is processed
+    Then the resulting file is:
+    """
+    Feature:
+
+      @test_case_1
+      Scenario Outline:
+        * a <param>
+      Examples:
+        | param | test_case_id |
+        | value | 1-1          |
+    """
+
+  Scenario: Outline, existing sub-ids
+
+  Note: Existing sub ids count against the indexes available for their associated parent id. If ids are mismatched (see
+  the feature on test case id validation), indexes are still assigned appropriately to avoid duplication.
+
+    Given the following feature file:
+    """
+    Feature:
+
+      @test_case_1
+      Scenario Outline:
+        * a <param>
+      Examples:
+        | param  | test_case_id |
+        | value1 | 1-1          |
+        | value2 |              |
+      Examples:
+        | param  | test_case_id |
+        | value1 | 1-3          |
+        | value2 |              |
+
+      Scenario Outline:
+        * a <param>
+      Examples: with a sub-id not associated with the parent test case
+        | param  | test_case_id |
+        | value1 | 8-1          |
+        | value2 |              |
+
+      @test_case_8
+      Scenario Outline: the outline that sub-id 8-1 should count against
+        * a <param>
+      Examples:
+        | param  | test_case_id |
+        | value2 |              |
+    """
+    When the file is processed
+    Then the resulting file is:
+    """
+    Feature:
+
+      @test_case_1
+      Scenario Outline:
+        * a <param>
+      Examples:
+        | param  | test_case_id |
+        | value1 | 1-1          |
+        | value2 | 1-4          |
+      Examples:
+        | param  | test_case_id |
+        | value1 | 1-3          |
+        | value2 | 1-5          |
+
+      @test_case_9
+      Scenario Outline:
+        * a <param>
+      Examples: with a sub-id not associated with the parent test case
+        | param  | test_case_id |
+        | value1 | 8-1          |
+        | value2 | 9-1          |
+
+      @test_case_8
+      Scenario Outline: the outline that sub-id 8-1 should count against
+        * a <param>
+      Examples:
+        | param  | test_case_id |
+        | value2 | 8-2          |
+    """
+
+  Scenario: Outline, explicit starting index
+
+  Note: Explicit starting indexes for sub-ids can be declared even if the parent id does not exist yet.
+
+  Note: By explicitly setting the starting index, there is no guarantee that the indexes used will not conflict with
+  existing indexes.
+
+    Given the following feature file:
+    """
+    Feature:
+
+      @test_case_1
+      Scenario Outline:
+        * a <param>
+      Examples:
+        | param  | test_case_id |
+        | value1 | 1-2          |
+        | value2 |              |
+        | value3 |              |
+
+      Scenario Outline:
+        * a <param>
+      Examples:
+        | param  | test_case_id |
+        | value1 | 2-1          |
+        | value2 |              |
+        | value3 |              |
+    """
+    And a start index of "2"
+    And a start index of "1" for testcase "1"
+    And a start index of "1" for testcase "2"
+    When the file is processed
+    Then the resulting file is:
+    """
+    Feature:
+
+      @test_case_1
+      Scenario Outline:
+        * a <param>
+      Examples:
+        | param  | test_case_id |
+        | value1 | 1-2          |
+        | value2 | 1-1          |
+        | value3 | 1-2          |
+
+      @test_case_2
+      Scenario Outline:
+        * a <param>
+      Examples:
+        | param  | test_case_id |
+        | value1 | 2-1          |
+        | value2 | 2-1          |
+        | value3 | 2-2          |
+    """
+
+  Scenario: Default indexing still applies to non-explicit indexes
+    Given the following feature file:
+    """
+    Feature:
+
+      Scenario Outline: Letting the default set the primary index
+        * a <param>
+      Examples:
+        | param  | test_case_id |
+        | value1 | 1-2          |
+        | value2 | 3-2          |
+        | value3 |              |
+
+      @test_case_2
+      Scenario Outline:
+        * a <param>
+      Examples:
+        | param  | test_case_id |
+        | value1 | 2-1          |
+        | value2 |              |
+        | value3 |              |
+    """
+    And a start index of "1" for testcase "2"
+    When the file is processed
+    Then the resulting file is:
+    """
+    Feature:
+
+      @test_case_3
+      Scenario Outline: Letting the default set the primary index
+        * a <param>
+      Examples:
+        | param  | test_case_id |
+        | value1 | 1-2          |
+        | value2 | 3-2          |
+        | value3 | 3-3          |
+
+      @test_case_2
+      Scenario Outline:
+        * a <param>
+      Examples:
+        | param  | test_case_id |
+        | value1 | 2-1          |
+        | value2 | 2-1          |
+        | value3 | 2-2          |
+    """

data/features/test_case_scanning.feature

@@ -0,0 +1,59 @@
+Feature: Scanning for tagged test cases
+
+  The test tagger is capable of finding all existing test objects within a test suite that have an id.
+
+
+  Scenario: Finds all tagged tests
+    Given the following feature file "file1.feature":
+    """
+    Feature:
+
+      Scenario:
+        * a step
+
+      @test_case_1
+      Scenario:
+        * a step
+
+      @test_case_7
+      Scenario Outline:
+        * a step
+      Examples: with rows
+        | test_case_id | param 1 |
+        | 7-1          | value 1 |
+      Examples: without rows
+        | test_case_id | param 1 |
+      Examples: with empty rows
+        | test_case_id | param 1 |
+        |              | value 1 |
+        | 7-2          | value 2 |
+      Examples: without an id parameter
+        | param 1 |
+        | value 1 |
+
+      #Missing the parent tag but the sub-tags still count
+      Scenario Outline:
+        * a step
+      Examples: with rows
+        | test_case_id | param 1 |
+        | 8-1          | value 1 |
+        | trash        | value 1 |
+    """
+    And the following feature file "file2.feature":
+    """
+    Feature: Just another feature to make sure that the entire suite is checked
+
+      @test_case_5
+      Scenario:
+        * a step
+    """
+    And a tag prefix of "@test_case_"
+    When the files are scanned
+    Then the following tagged test objects are found:
+      | path/to/file1.feature:7  |
+      | path/to/file1.feature:11 |
+      | path/to/file1.feature:15 |
+      | path/to/file1.feature:21 |
+      | path/to/file1.feature:31 |
+      | path/to/file1.feature:32 |
+      | path/to/file2.feature:4  |

data/features/test_case_scanning_payload.feature

@@ -0,0 +1,32 @@
+Feature: Scan results return useful information
+
+  Test objects are returned in the results of a scan.
+
+  Scenario: Scanned scenario
+    Given the following feature file "file1.feature":
+    """
+    Feature:
+
+      @test_case_1
+      Scenario:
+        * a step
+    """
+    And a tag prefix of "@test_case_"
+    When the files are scanned
+    Then the payload is a test object
+
+  Scenario: Scanned outline
+    Given the following feature file "test_file.feature":
+    """
+    Feature:
+
+      @test_case_7
+      Scenario Outline:
+        * a step
+      Examples: with rows
+        | test_case_id | param 1 |
+        | 7-1          | value 1 |
+    """
+    And a tag prefix of "@test_case_"
+    When the files are scanned
+    Then the payload has a test and a test row

data/features/test_case_tagging.feature

@@ -0,0 +1,193 @@
+Feature: Tagging test cases for uniqueness
+
+  Tests are tagged based on a simple <prefix><unique_id> pattern. Every test gets a tag at the test level and, in the case outlines, a sub-id for each of its example rows. Any test which has already been given an id tag or other id information will not have that information modified by the tagging process.
+
+
+  Background:
+    * a tag prefix of "@test_case_"
+
+
+  Scenario: Tagging tests that don't have id tags
+
+  This is one of the most common cases, wherein new tests have been written but they have not yet been tagged by the tagging tool.
+
+    Given the following feature file:
+      """
+      Feature:
+
+        Scenario:
+          * a step
+
+        Scenario Outline:
+          * a step
+        Examples: with rows
+          | param 1 |
+          | value 1 |
+        Examples: without rows
+          | param 1 |
+      """
+    When the file is processed
+    Then the resulting file is:
+      """
+      Feature:
+
+        @test_case_1
+        Scenario:
+          * a step
+
+        @test_case_2
+        Scenario Outline:
+          * a step
+        Examples: with rows
+          | param 1 | test_case_id |
+          | value 1 | 2-1          |
+        Examples: without rows
+          | param 1 | test_case_id |
+      """
+
+  Scenario: Tagging tests that already have id tags
+
+  This is the other most common case, wherein existing tests have already been tagged by the tagging tool.
+
+    Given the following feature file:
+      """
+      Feature:
+
+        @test_case_1
+        Scenario:
+          * a step
+
+        @test_case_2
+        Scenario Outline:
+          * a step
+        Examples: with rows
+          | param 1 | test_case_id |
+          | value 1 | 2-1          |
+        Examples: without rows
+          | param 1 | test_case_id |
+      """
+    When the file is processed
+    Then the resulting file is:
+      """
+      Feature:
+
+        @test_case_1
+        Scenario:
+          * a step
+
+        @test_case_2
+        Scenario Outline:
+          * a step
+        Examples: with rows
+          | param 1 | test_case_id |
+          | value 1 | 2-1          |
+        Examples: without rows
+          | param 1 | test_case_id |
+      """
+
+
+  Scenario: Tagging outlines that are partially 'filled in'
+
+  Whereas scenarios either have an id tag or don't, outlines have many more places for identifying information to be placed and it is quite possible that some of that information is already present. Maybe several scenarios were combined into one outline and an id tag got carried over. Maybe an extra row or entire example table was added to an already tagged outline. Maybe someone removed id information by accident. In any case, the missing information is replaced by the tagging tool.
+
+    Given the following feature file:
+      """
+      Feature:
+
+        @test_case_1
+        Scenario Outline: Has an id tag, but no sub-ids
+          * a step
+        Examples: with rows
+          | param 1 |
+          | value 1 |
+        Examples: without rows
+          | param 1 |
+
+        Scenario Outline: Sub-ids present but no top level tag
+          * a step
+        Examples: Missing row ids
+          | param 1 | test_case_id |
+          | value 2 |              |
+          | value 1 | some_id      |
+        Examples: Already has id column
+          | param 1 | test_case_id |
+        Examples: Doesn't have id column
+          | param 1 |
+          | value 3 |
+      """
+    When the file is processed
+    Then the resulting file is:
+      """
+      Feature:
+
+        @test_case_1
+        Scenario Outline: Has an id tag, but no sub-ids
+          * a step
+        Examples: with rows
+          | param 1 | test_case_id |
+          | value 1 | 1-1          |
+        Examples: without rows
+          | param 1 | test_case_id |
+
+        @test_case_2
+        Scenario Outline: Sub-ids present but no top level tag
+          * a step
+        Examples: Missing row ids
+          | param 1 | test_case_id |
+          | value 2 | 2-1          |
+          | value 1 | some_id      |
+        Examples: Already has id column
+          | param 1 | test_case_id |
+        Examples: Doesn't have id column
+          | param 1 | test_case_id |
+          | value 3 | 2-2          |
+      """
+
+  Scenario: Tagging a multi-file test suite
+    Given the following feature file:
+      """
+      Feature:
+
+        Scenario:
+          * a step
+      """
+    And the following feature file:
+      """
+      Feature:
+
+        Scenario Outline:
+          * a step
+        Examples: with rows
+          | param 1 |
+          | value 1 |
+        Examples: without rows
+          | param 1 |
+        Examples: some more rows
+          | param 1 |
+          | value 1 |
+      """
+    When the files are processed
+    Then the resulting first file is:
+      """
+      Feature:
+
+        @test_case_1
+        Scenario:
+          * a step
+      """
+    And the resulting second file is:
+      """
+      Feature:
+
+        @test_case_2
+        Scenario Outline:
+          * a step
+        Examples: with rows
+          | param 1 | test_case_id |
+          | value 1 | 2-1          |
+        Examples: without rows
+          | param 1 | test_case_id |
+        Examples: some more rows
+          | param 1 | test_case_id |
+          | value 1 | 2-2          |
+      """