cucumber_analytics 0.0.1

data/.gitignore

@@ -0,0 +1,17 @@
+*.gem
+*.rbc
+.bundle
+.config
+.yardoc
+Gemfile.lock
+InstalledFiles
+_yardoc
+coverage
+doc/
+lib/bundler/man
+pkg
+rdoc
+spec/reports
+test/tmp
+test/version_tmp
+tmp

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in cucumber-analytics.gemspec
+gemspec

data/LICENSE

@@ -0,0 +1,22 @@
+Copyright (c) 2012 Eric Kessler
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.rdoc

@@ -0,0 +1,76 @@
+= CucumberAnalytics
+
+The intention of this gem is to provide a useful mechanism by which to answer
+all of the burning questions that one might have about their Cucumber test base.
+
+* Did old features get removed and leave dead step definitions behind? Or
+perhaps the other way around and half of the steps in the code base are
+undefined?
+
+* Are there duplicate scenarios strewn about everywhere that only differ in
+their step arguments and could be reduced into a single outline?
+
+* Just how many different tags *are* there in this thing?
+
+These are the kinds of questions that this gem aims to make answering easy,
+either through directly providing the answer or by providing sufficient tools
+that determining the answer yourself is just a few straightforward object
+manipulations away.
+
+== Installation
+
+Add this line to your application's Gemfile:
+
+    gem 'cucumber_analytics'
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install cucumber_analytics
+
+== Usage
+
+First things first. Load up the gem code.
+
+  require 'cucumber_analytics'
+
+Next, let's generate a model for our Cucumber suite.
+
+  directory = CucumberAnalytics::ParsedDirectory.new('path/to/the/code_directory')
+  world = CucumberAnalytics::World
+
+Now it's time to take a look around and see what we can see.
+
+  all_tags = world.tags_in(directory)
+  puts all_tags
+  #=> ["@Unit", "@Fragile", "@wip", "@wip", "@Critical", "@Unit", "@wip", "@Deprecated", "@wip", "@wip"]
+
+  puts all_tags.uniq
+  #=> ["@Unit", "@Fragile", "@wip", "@Critical", "@Deprecated"]
+
+  wip_tags = all_tags.select{ |tag| tag == '@wip' }
+
+  puts wip_tags.count.to_f / all_tags.count
+  #=> 0.5
+
+
+  all_steps = world.steps_in(directory)
+  puts all_steps.collect{ |step| step.step_text(with_keywords: false)}
+  #=> ["some step", "the user logs in", "the user will log in", "another step", "the user "Bob" logs in"]
+
+So with a few simple commands we have discovered that there are five different
+tags in our codebase and that @wip tags account for half of all usages. We have
+also discovered that our team is creating several redundant steps that could be
+rewritten into a single, reusable step.
+
+
+== Contributing
+
+1. Fork it
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Added some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create new Pull Request

data/Rakefile

@@ -0,0 +1,2 @@
+#!/usr/bin/env rake
+require "bundler/gem_tasks"

data/cucumber_analytics.gemspec

@@ -0,0 +1,22 @@
+# -*- encoding: utf-8 -*-
+require File.expand_path('../lib/cucumber_analytics/version', __FILE__)
+
+Gem::Specification.new do |gem|
+  gem.authors       = ["Eric Kessler"]
+  gem.email         = ["morrow748@gmail.com"]
+  gem.description   = %q{Static analysis of Cucumber tests made easy.}
+  gem.summary       = %q{This gem provides an API to programmatically break down Cucumber feature files so that they can be inspected and analyzed in a straightforward manner.}
+  gem.homepage      = "https://github.com/enkessler/cucumber_analytics"
+
+  gem.files         = `git ls-files`.split($\)
+  gem.executables   = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
+  gem.test_files    = gem.files.grep(%r{^(test|spec|features)/})
+  gem.name          = "cucumber_analytics"
+  gem.require_paths = ["lib"]
+  gem.version       = CucumberAnalytics::VERSION
+
+  gem.add_development_dependency("rake")
+  gem.add_development_dependency("cucumber")
+  gem.add_development_dependency("wrong")
+  gem.add_development_dependency("simplecov")
+end

data/features/analysis/feature_collection.feature

@@ -0,0 +1,39 @@
+Feature: Features can be collected from arbitrary parts of the codebase.
+
+
+  Acceptance criteria
+
+  Features can be collected from:
+    1. files
+    2. directories
+
+
+  Background: Setup test codebase
+    Given a directory "feature_directory"
+    And the following feature file "test_file_1.feature":
+    """
+    Feature: The test feature 1.
+    """
+    And the file "test_file_1.feature" is read
+    And a directory "feature_directory/nested_directory"
+    And the following feature file "test_file_2.feature":
+    """
+    Feature: The test feature 2.
+    """
+    And the file "test_file_2.feature" is read
+    When the directory "feature_directory" is read
+    And the directory "feature_directory/nested_directory" is read
+
+
+  Scenario: Features can be collected from files
+    Then the features collected from file "1" are as follows:
+      | The test feature 1. |
+    Then the features collected from file "2" are as follows:
+      | The test feature 2. |
+
+  Scenario: Features can be collected from directories
+    Then the features collected from directory "1" are as follows:
+      | The test feature 1. |
+      | The test feature 2. |
+    And the features collected from directory "2" are as follows:
+      | The test feature 2. |

data/features/analysis/feature_file_collection.feature

@@ -0,0 +1,36 @@
+Feature: Feature files can be collected from arbitrary parts of the codebase.
+
+
+  Acceptance criteria
+
+  Feature files can be collected from:
+    1. directories
+
+
+  Background: Setup test codebase
+    Given a directory "feature_directory"
+    And a directory "feature_directory/nested_directory_1"
+    And the following feature file "test_file_1.feature":
+    """
+    Feature: The test feature 1.
+    """
+    And the file "test_file_1.feature" is read
+    And a directory "feature_directory/nested_directory_2"
+    And the following feature file "test_file_2.feature":
+    """
+    Feature: The test feature 2.
+    """
+    And the file "test_file_2.feature" is read
+    When the directory "feature_directory" is read
+    And the directory "feature_directory/nested_directory_1" is read
+    And the directory "feature_directory/nested_directory_2" is read
+
+
+  Scenario: Feature files can be collected from directories
+    Then the files collected from directory "1" are as follows:
+      | test_file_1.feature |
+      | test_file_2.feature |
+    And the files collected from directory "2" are as follows:
+      | test_file_1.feature |
+    And the files collected from directory "3" are as follows:
+      | test_file_2.feature |

data/features/analysis/step_collection.feature

@@ -0,0 +1,137 @@
+Feature: Steps can be collected from arbitrary parts of the codebase.
+
+
+  Acceptance criteria
+
+  Steps (both defined and undefined) can be collected from:
+    1. backgrounds
+    2. scenarios
+    3. outlines
+    4. features
+    5. files
+    6. directories
+
+
+  Background: Setup test directories
+    Given a directory "feature_directory"
+    And the following feature file "test_file_1.feature":
+    """
+    Feature: The test feature 1.
+
+      Background: Some general test setup stuff.
+        Given a defined step
+        And an undefined step
+
+      Scenario: The scenario's name.
+        Given another defined step
+        Then another undefined step
+    """
+    And the file "test_file_1.feature" is read
+    And a directory "feature_directory/nested_directory"
+    And the following feature file "test_file_2.feature":
+    """
+    Feature: The test feature 2.
+
+       Scenario Outline: The scenario outline's name.
+        Given a defined step
+        When another defined step
+        Then *<this>* *step is* *<undefined>*
+      Examples:
+        | this | undefined |
+        | x    | y         |
+      Examples:
+        | this | undefined |
+        | a    | b         |
+    """
+    And the file "test_file_2.feature" is read
+    And the following step definition file "some_step_defs.rb":
+    """
+    Given /^a defined step$/ do
+      pending
+    end
+    """
+    And the following step definition file "more_step_defs.rb":
+    """
+    Given /^another defined step$/ do
+      pending
+    end
+    """
+    When the step definition file "some_step_defs.rb" is read
+    And the step definition file "more_step_defs.rb" is read
+    And the directory "feature_directory" is read
+
+
+
+  Scenario: Steps can be collected from backgrounds
+    Then the steps collected from feature "1" background are as follows:
+      | Given a defined step  |
+      | And an undefined step |
+    And the "defined" steps collected from feature "1" background are as follows:
+      | Given a defined step |
+    And the "undefined" steps collected from feature "1" background are as follows:
+      | And an undefined step |
+
+  Scenario: Steps can be collected from scenarios
+    Then the steps collected from feature "1" test "1" are as follows:
+      | Given another defined step  |
+      | Then another undefined step |
+    And the "defined" steps collected from feature "1" test "1" are as follows:
+      | Given another defined step |
+    And the "undefined" steps collected from feature "1" test "1" are as follows:
+      | Then another undefined step |
+
+  Scenario: Steps can be collected from scenario outlines
+    Then the steps collected from feature "2" test "1" are as follows:
+      | Given a defined step                  |
+      | When another defined step             |
+      | Then *<this>* *step is* *<undefined>* |
+    And the "defined" steps collected from feature "2" test "1" are as follows:
+      | Given a defined step      |
+      | When another defined step |
+    And the "undefined" steps collected from feature "2" test "1" are as follows:
+      | Then *<this>* *step is* *<undefined>* |
+
+  Scenario: Steps can be collected from features
+    Then the steps collected from feature "1" are as follows:
+      | Given a defined step        |
+      | And an undefined step       |
+      | Given another defined step  |
+      | Then another undefined step |
+    And the "defined" steps collected from feature "1" are as follows:
+      | Given a defined step       |
+      | Given another defined step |
+    And the "undefined" steps collected from feature "1" are as follows:
+      | And an undefined step       |
+      | Then another undefined step |
+
+  Scenario: Steps can be collected from files
+    Then the steps collected from file "1" are as follows:
+      | Given a defined step        |
+      | And an undefined step       |
+      | Given another defined step  |
+      | Then another undefined step |
+    And the "defined" steps collected from file "1" are as follows:
+      | Given a defined step       |
+      | Given another defined step |
+    And the "undefined" steps collected from file "1" are as follows:
+      | And an undefined step       |
+      | Then another undefined step |
+
+  Scenario: Steps can be collected from directories
+    Then the steps collected from the directory are as follows:
+      | Given a defined step                  |
+      | And an undefined step                 |
+      | Given another defined step            |
+      | Then another undefined step           |
+      | Given a defined step                  |
+      | When another defined step             |
+      | Then *<this>* *step is* *<undefined>* |
+    And the "defined" steps collected from the directory are as follows:
+      | Given a defined step       |
+      | Given another defined step |
+      | Given a defined step       |
+      | When another defined step  |
+    And the "undefined" steps collected from the directory are as follows:
+      | And an undefined step                 |
+      | Then another undefined step           |
+      | Then *<this>* *step is* *<undefined>* |

data/features/analysis/tag_collection.feature

@@ -0,0 +1,91 @@
+Feature: Tags can be collected from arbitrary parts of the codebase.
+
+
+  Acceptance criteria
+
+  Tags can be collected from:
+    1. scenarios
+    2. outlines
+    3. features
+    4. files
+    5. directories
+
+
+  Background: Setup test codebase
+    Given a directory "feature_directory"
+    And the following feature file "test_file_1.feature":
+    """
+    @feature_tag_1
+    Feature: The test feature 1.
+
+      @scenario_tag_1
+      Scenario: The first scenario's name.
+        Given the first step
+        When the second step
+        Then the third step
+
+      @outline_tag_1
+      Scenario Outline: The scenario outline's name.
+        Given the first "<param1>"
+        When the second "<param2>"
+        Then the third step
+      @example_tag_1
+      Examples: text describing the significance of the examples
+        | param1 | param2 |
+        | x      | y      |
+      @example_tag_2
+      Examples: some examples with different significance and a tag
+        | param1 | param2 |
+        | a      | b      |
+    """
+    And the file "test_file_1.feature" is read
+    And a directory "feature_directory/nested_directory"
+    And the following feature file "test_file_2.feature":
+    """
+    @feature_tag_2
+    Feature: The test feature 2.
+
+      @scenario_tag_2
+      Scenario: The first scenario's name.
+        Given the first step
+        When the second step
+        Then the third step
+    """
+    And the file "test_file_2.feature" is read
+    When the directory "feature_directory" is read
+
+  Scenario: Tags can be collected from scenarios
+    Then the tags collected from feature "1" test "1" are as follows:
+      | @scenario_tag_1 |
+
+  Scenario: Tags can be collected from scenario outlines
+    Then the tags collected from feature "1" test "2" are as follows:
+      | @outline_tag_1 |
+      | @example_tag_1 |
+      | @example_tag_2 |
+
+  Scenario: Tags can be collected from features
+    Then the tags collected from feature "1" are as follows:
+      | @feature_tag_1  |
+      | @scenario_tag_1 |
+      | @outline_tag_1  |
+      | @example_tag_1  |
+      | @example_tag_2  |
+
+  Scenario: Tags can be collected from files
+    Then the tags collected from file "1" are as follows:
+      | @feature_tag_1  |
+      | @scenario_tag_1 |
+      | @outline_tag_1  |
+      | @example_tag_1  |
+      | @example_tag_2  |
+
+  Scenario: Tags can be collected from directories
+    Then the tags collected from directory are as follows:
+      | @feature_tag_1  |
+      | @scenario_tag_1 |
+      | @outline_tag_1  |
+      | @example_tag_1  |
+      | @example_tag_2  |
+      | @feature_tag_2  |
+      | @scenario_tag_2 |

data/features/analysis/test_collection.feature

@@ -0,0 +1,69 @@
+Feature: Tests can be collected from arbitrary parts of the codebase.
+
+
+  Acceptance criteria
+
+  Tests can be collected from:
+    1. features
+    2. files
+    3. directories
+
+
+  Background: Setup test codebase
+    Given a directory "feature_directory"
+    And the following feature file "test_file_1.feature":
+    """
+    Feature: The test feature 1.
+
+      Scenario: Test 1
+        Given the first step
+        When the second step
+        Then the third step
+
+      Scenario Outline: Test 2
+        Given the first "<param1>"
+        When the second "<param2>"
+        Then the third step
+      Examples: text describing the significance of the examples
+        | param1 | param2 |
+        | x      | y      |
+      Examples: some examples with different significance and a tag
+        | param1 | param2 |
+        | a      | b      |
+    """
+    And the file "test_file_1.feature" is read
+    And a directory "feature_directory/nested_directory"
+    And the following feature file "test_file_2.feature":
+    """
+    Feature: The test feature 2.
+
+      Scenario: Test 3
+        Given the first step
+        When the second step
+        Then the third step
+    """
+    And the file "test_file_2.feature" is read
+    When the directory "feature_directory" is read
+    And the directory "feature_directory/nested_directory" is read
+
+  Scenario: Tests can be collected from features
+    Then the tests collected from feature "1" are as follows:
+      | Test 1 |
+      | Test 2 |
+    Then the tests collected from feature "2" are as follows:
+      | Test 3 |
+
+  Scenario: Tests can be collected from files
+    Then the tests collected from file "1" are as follows:
+      | Test 1 |
+      | Test 2 |
+    Then the tests collected from file "2" are as follows:
+      | Test 3 |
+
+  Scenario: Tests can be collected from directories
+    Then the tests collected from directory "1" are as follows:
+      | Test 1 |
+      | Test 2 |
+      | Test 3 |
+    And the tests collected from directory "2" are as follows:
+      | Test 3 |

data/features/analysis/test_comparison.feature

@@ -0,0 +1,123 @@
+Feature: Test equality can be determined.
+
+
+  Acceptance criteria
+
+  Tests can be compared for equality.
+    1. tests whose steps are the same except for arguments and keywords (i.e.
+       they match the same step definition) are equal
+
+
+  Background: Test file setup.
+    Given the following feature file:
+    """
+    Feature: A feature with duplicate tests.
+
+      Scenario: A test
+        Given this *parameterized* step takes a table:
+          | data 1 |
+          | data 2 |
+        And some setup step
+        When a step with a *parameter*
+        And a big step:
+        \"\"\"
+        little doc block
+        \"\"\"
+        Then *lots* *of* *parameters*
+
+      Scenario: Same test, different arguments and keywords
+        Given this *similarly parameterized* step takes a table:
+          | data 3 |
+          | data 4 |
+        Given some setup step
+        When a step with a *parameter*
+        * a big step:
+        \"\"\"
+        A little
+        bigger doc block
+        \"\"\"
+        Then *lots* *of* *parameters*
+
+      Scenario Outline: This is the same test as an outline
+        Given this *parameterized* step takes a table:
+          | <param1> |
+          | <param2> |
+        And some setup step
+        When a step with a *parameter*
+        And a big step:
+        \"\"\"
+        little doc block
+        \"\"\"
+        Then *lots* *of* *parameters*
+      Examples:
+        | param1 | param2 |
+        | x      | y      |
+      Examples:
+        | param1 | param2 |
+        | a      | b      |
+
+      Scenario Outline: Same outline, different arguments and keywords
+        Given this *similarly parameterized* step takes a table:
+          | <param3> |
+          | <param4> |
+        Given some setup step
+        When a step with a *slightly different parameter*
+        * a big step:
+        \"\"\"
+        A little
+        bigger doc block
+        \"\"\"
+        Then *lots* *of effectively the same* *parameters*
+      Examples:
+        | param1 | param2 |
+        | h      | k      |
+      Examples:
+        | param1 | param2 |
+        | i      | j      |
+
+      Scenario: A different test
+        Given this *parameterized* step takes a table:
+          | data 1 |
+          | data 2 |
+        And not the same setup step as before
+        When a step with a *parameter*
+        And a big step:
+        \"\"\"
+        little doc block
+        \"\"\"
+        Then *lots* *of* *parameters*
+
+      Scenario Outline: This is the same different test as an outline
+        Given this *similarly parameterized* step takes a table:
+          | <param1> |
+          | <param2> |
+        And not the same setup step as before
+        When a step with a *slightly different parameter*
+        And a big step:
+        \"\"\"
+        A little
+        bigger doc block
+        \"\"\"
+        Then *lots* *of effectively the same* *parameters*
+      Examples:
+        | param1 | param2 |
+        | x      | y      |
+      Examples:
+        | param1 | param2 |
+        | a      | b      |
+    """
+    And parameter delimiters of "*" and "*"
+    When the file is read
+
+
+  Scenario: Scenario to Scenario comparison
+    Then test "1" is equal to test "2"
+    And test "1" is not equal to test "5"
+
+  Scenario: Outline to Outline comparison
+    Then test "3" is equal to test "4"
+    And test "3" is not equal to test "6"
+
+  Scenario: Scenario to Outline comparison
+    Then test "1" is equal to test "3"
+    And test "1" is not equal to test "6"