cucumber_testrail 1.3.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 1ecd80cf8e65a0501a6682cbf5dc0d3673711927
+  data.tar.gz: 2ff6834a864a02c9e85ea4c0a27be6af917dcb18
+SHA512:
+  metadata.gz: 96c2ea59b4b6b57a8d94b73a928e06660cdd23f67ceb53b5f37d0c251eb9679a5b551ae3dde621fc9da4c04aef3b0dbfe79a6e51cd704b3bb247bbc2c55db4d4
+  data.tar.gz: e369fa2f42664c73d8de0f94dc5e85fa020ad0979799c99dc3abcb698da5eef6d994a998285ea524130a474be3a0cf3d9d613c6a6264090b480ed62f27439d37

data/.gitignore

@@ -0,0 +1,24 @@
+*.gem
+*.rbc
+.bundle
+.config
+.yardoc
+Gemfile.lock
+InstalledFiles
+_yardoc
+coverage
+doc/
+lib/bundler/man
+pkg
+rdoc
+spec/reports
+test/tmp
+test/version_tmp
+tmp
+*.bundle
+*.so
+*.o
+*.a
+mkmf.log
+*.swp
+

data/.ruby-gemset

@@ -0,0 +1 @@
+cuke_rail

data/.ruby-version

@@ -0,0 +1 @@
+ruby-2.1.2

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in cucumber_testrail.gemspec
+gemspec

data/LICENSE.txt

@@ -0,0 +1,22 @@
+Copyright (c) 2014 John Small
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,85 @@
+# CucumberTestrail
+
+## Overview
+
+This gem helps you synchronise your Cucumber test suites into Testrail. It uses tags to link Testrail projects, test suites, testcases
+and write reports for testruns into Testrail. It will also create new testcases if it can't find them in Testrail and update your feature files
+to add the tag to the scenario. It writes the step definition into the testcase and keeps it updated.
+
+The central idea is that tags in the features files are used to link your scenarios to Testrail. The identifiers used are 'project','suite','testcase','testrun' they are all looked up
+from tags on the scenario, tags on the feature or environment variables. E.g. on the scenario you might have a tag **@testcase_99**, this will link to the Testrail tescase
+with id=99. 
+
+In Testrail, tests are instances of testcases within the context of a testrun. Therefore to report on a test instance you need a combination of testcase
+and testrun. The testrun can be one of a tag at scenario level, a tag at feature level or and environment level, e.g. a tag **@testrun_5** or 
+environment variable **TESTRUN=5**. The combination of testcase and testrun form a Testrail test and the result of the senario is add to a list of results for the Testrail test. How you allocate test suites, testcases and testruns is entirely up to you.
+
+## Installation
+The gem is hosted our own gemserver at http://ec2-54-77-102-63.eu-west-1.compute.amazonaws.com:9292 so you need to add these lines to your Gemfile
+
+
+Add this line to your application's Gemfile:
+
+    source "http://ec2-54-77-102-63.eu-west-1.compute.amazonaws.com:9292"
+    gem 'cucumber_testrail',"~>1.1.0" #whatever is the latest version
+
+    $ bundle
+
+### Basic usage
+
+Once you've installed the gem and bundled it, you need to add the methods in the gem to the World object. In **env.rb** add
+
+     World CucumberTestrail
+
+Then you need to add the code to After scenario hooks
+
+     send_result(scenario)
+
+Then state which testrun you're operating in. This can be either a tag e.g. **@testrun_5** at the scenario or feature level, or in the environment **TESTRUN=5**
+
+Set up environment variables for the Testrail URL, username and password
+
+     export TESTRAIL__USER="your-testrail-user"
+     export TESTRAIL_PASSWORD="your-testrail-password"
+     export TESTRAIL_BASE_URL="your-testrail-server"
+
+
+#### What if I don't have a testcase id? 
+
+The gem will try to find one that matches the title and if it can't it'll create a new testcase. It then takes the testcase id and adds a tag to your 
+feature file. That requires at least one line in between each scenario, but you do that anyway.
+
+To create the testcase it needs to know the project id suite id and the sub section id. 
+Those should be added as a tags at feature level e.g. **@project_2 @suite_10 @sub_section_14**
+
+#### Outlines!!
+
+Each line in the Examples section of a Scenario Outline will be mapped to a unique testcase using the scenario title, and the named variables in the
+examples. e.g.
+
+     Scenario Outline: A title
+       Given some steps
+       Examples: some example lines
+        |heading 1 | heading 2 |
+        | ex1      | ex2       |
+
+Will be mapped to a testcase with title "A title : heading 1='ex1', heading 2='ex2'". Because we will have many testcases for a single scenario outline
+then the gem doesn't add testcase tags 
+
+#### What if I don't want to send a report to Testrail
+
+There's a special tag **@ignore_testrail** and environment variable **IGNORE_TESTRAIL=true**. So you can specify one scenario should not be linked to Testrail,
+one feature file, or the whole run.
+
+#### What if I want to tag it with a jira ticket number?
+
+Add the tag **@jira_STORE-123** for example. This will then appear in the defect section of the test report as STORE-123. You can add multiple jira tags to a scenario
+
+
+### Synchronising Testrail testcases to Cucumber scenarios?
+
+The gem relies first and foremost on the tag **@testcase_id**. If that tag is present it will use that id without further checks. When it writes the result of the
+test it also updates the title of the testcase to match the title of the scenario and the contents of the testcase steps to match the scenario steps. It overwrites 
+these without any checks. That means if you have a Testrail testcase 5, with title 'Check that the sun comes up in the morning' and you give a tag **@testcase_5**
+to a scenario titled 'The user should be logged in', then at the end of the run that Testrail testcase will now have the title taken from the scenario.
+

data/Rakefile

@@ -0,0 +1,4 @@
+require "bundler/gem_tasks"
+Dir.glob('tasks/**/*.rake').each(&method(:import))
+task :default => :spec
+

data/cucumber_testrail.gemspec

@@ -0,0 +1,30 @@
+# coding: utf-8
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'cucumber_testrail/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = "cucumber_testrail"
+  spec.version       = CucumberTestrail::VERSION
+  spec.authors       = ["John Small"]
+  spec.email         = ["john.small@bbc.com"]
+  spec.summary       = %q{Sync Cucumber specs and results to TestRail}
+  spec.description   = %q{Call cucumber_testrail.send_result(scenario) in an After hook. It will match existing testcases by scenario title or add new testcases
+  and then add the result of the run to TestRail. See the README.md}
+  spec.license       = "MIT"
+
+  spec.files         = `git ls-files -z`.split("\x0")
+  spec.executables   = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
+  spec.test_files    = spec.files.grep(%r{^(test|spec|features)/})
+  spec.require_paths = ["lib"]
+
+  spec.add_runtime_dependency "cucumber","~>1.3"
+
+  spec.add_development_dependency "bundler", "~> 1.6"
+  spec.add_development_dependency "rake","~>10.3"
+  spec.add_development_dependency "rspec","~>3.0"
+  spec.add_development_dependency "vcr","~>2.9"
+  spec.add_development_dependency "webmock","~>1.18"
+  spec.add_development_dependency "rdoc","~>4.1"
+  spec.add_development_dependency "geminabox","~>0.12"
+end

data/lib/cucumber_testrail.rb

@@ -0,0 +1,7 @@
+require "cucumber_testrail/version"
+require "cucumber_testrail/reopen_scenario"
+require "cucumber_testrail/testrail"
+require "cucumber_testrail/external_reporting"
+module CucumberTestrail
+  include Report
+end

data/lib/cucumber_testrail/external_reporting.rb

@@ -0,0 +1,106 @@
+require 'tempfile'
+require 'fileutils'
+module Report 
+
+  ##
+  # allows access to an instance of the testrail object. If it's not set then it will be created on the first read
+  # most useful when testing because you can put a spy in place of the testrail api to check the expected calls are being made
+  attr_accessor :testrail
+
+  ## The only entry point from your Cucumber After scenario hooks.
+  # You need to have this code ;-
+  #
+  # After do |scenario|
+  #   send_result(scenario)
+  # end
+  #
+  # It returns true if it wrote to Testrail and false if it didn't. I.e. if you set *ignore_testrail* then it won't send results to Testrail and it will return false
+  #
+  def send_result(scenario) 
+    if scenario.ignore_testrail?
+      result = false
+    else
+      external_reference = get_an_external_reference(scenario)
+      raise 'duff external reference' unless external_reference.to_i > 0
+      #p "external reference #{external_reference}"
+      send_test_result_to_testrail(scenario,external_reference) unless scenario.skip_result?
+      update_test_case(scenario,external_reference)
+      # only write the testcase tag if it's a scenario, not a scenario outline
+      update_source_file(scenario,"testcase_#{external_reference}") if scenario.is_a?(Cucumber::Ast::Scenario)
+      result = true
+    end
+    result
+  end
+
+
+  ##
+  # This method takes an external reference, e.g. testcase_119 and adds it as a tag before the scenario
+  #
+  def update_source_file(scenario, external_reference)
+    #this could be done on one line with sed. But the format is different on Mac OS and GNU Linux version and it definitely won't work on a Windows machine
+    path = scenario.file
+    lines = IO.readlines(scenario.file)
+    lines[scenario.tag_line].gsub!(/@testcase_(\d*)|\n/," @#{external_reference}") unless lines[scenario.tag_line] =~/#{external_reference}/
+    temp_file = Tempfile.new('foo')
+    begin
+      File.open(path, 'r') do |file|
+        lines.each do |line|
+          temp_file.puts line
+        end
+      end
+      temp_file.close
+      FileUtils.mv(temp_file.path, path)
+    ensure
+      temp_file.close
+      temp_file.unlink
+    end
+  end
+
+  ##
+  # If we don't have a tag testcase id then we need to find one. Either by matching a testcase by title, or by creating a new testcase
+  # The input is a scenario and the result is a testcase id
+  def get_an_external_reference(scenario)
+    (check_if_a_test_case_id_exists(scenario) || locate_a_test_case_by_name(scenario) || create_a_test_case_from_scenario(scenario)['id']).to_i
+  end
+
+  def create_a_test_case_from_scenario(scenario)
+    testrail.send_post("add_case/#{scenario.sub_section || scenario.suite}",scenario.testrail_testcase)
+  end
+
+  def locate_a_test_case_by_name(scenario)
+    if scenario.sub_section
+      restrict_by ="#{scenario.project}&suite_id=#{scenario.suite}&section_id=#{scenario.sub_section}"
+    else
+      restrict_by ="#{scenario.project}&suite_id=#{scenario.suite}"
+    end
+    matches = testrail.send_get("get_cases/#{restrict_by}").select{|m| m['title'] == scenario.title}
+    unless matches.empty?
+      matches.first['id']
+    end  
+  end
+
+  def check_if_a_test_case_id_exists(scenario)
+    if scenario.testcase
+      matches = testrail.send_get("get_cases/#{scenario.project}&suite_id=#{scenario.suite}").select{|m| m['id'].to_i == scenario.testcase.to_i}
+      ref = matches.first['id']
+    end
+    ref
+  end
+
+  def send_test_result_to_testrail(scenario,external_reference=nil)
+    testcase = external_reference || scenario.testcase
+    testrail.send_post("add_result_for_case/#{scenario.testrun}/#{testcase}",scenario.testrail_test_report)
+  end
+
+  def update_test_case(scenario,external_reference)
+    testcase = external_reference || scenario.testcase
+    testrail.send_post("update_case/#{testcase}",scenario.testrail_testcase)
+  end
+
+  def testrail
+    raise 'TESTRAIL_BASE_URL must be set up in your environment' unless ENV['TESTRAIL_BASE_URL']
+    raise 'TESTRAIL_USER must be set up in your environment' unless ENV['TESTRAIL_USER']
+    raise 'TESTRAIL_PASSWORD must be set up in your environment' unless ENV['TESTRAIL_PASSWORD']
+    @testrail ||= TestRail::APIClient.new(ENV['TESTRAIL_BASE_URL'],ENV['TESTRAIL_USER'],ENV['TESTRAIL_PASSWORD'])
+  end   
+end

data/lib/cucumber_testrail/reopen_scenario.rb

@@ -0,0 +1,58 @@
+require_relative 'testrail_extensions'
+module Cucumber #:nodoc: 
+  module Ast #:nodoc:
+    class Scenario
+      include Testrail::Scenario
+      ##
+      # return the steps and as strings ready to write to the testcase
+      def steps_as_string
+        to_sexp.select{|a| a[0]==:step_invocation}.map do |s|
+          if s[4]
+            ["#{s[2]}#{s[3]}"]+s[4].select{|e| e.is_a?(Array) && e[0]==:row}.map do |l|
+              "| #{l.select{|e| e.is_a?(Array)}.map{|e| e[1]}.join(' | ')} |"
+            end
+          else  
+            "#{s[2]}#{s[3]}"
+          end
+        end.flatten.join("\n")
+      end
+
+      def tag_line
+        line - 2
+      end
+    end
+
+    class ScenarioOutline
+      def my_steps
+        @steps ||= @background.step_collection(step_invocations)
+      end
+    end
+
+    class OutlineTable
+      class ExampleRow
+        include Testrail::Scenario
+
+        def feature_tags
+          scenario_outline.feature_tags
+        end
+
+        def title
+          line = self.to_hash.to_a.map{|a| "#{a[0]}='#{a[1]}'"}.join(', ')
+          "#{scenario_outline.title} :: #{line}"
+        end  
+
+        def steps_as_string
+          #example rows in an outline table don't respond to steps so we have to check the scenario_outline
+          scenario_outline.raw_steps.map{|s| "#{s.keyword} #{s.name}"}.join("\n")
+        end
+
+        def tag_line
+          scenario_outline.file_colon_line.split(':')[1].to_i - 2
+        end
+        def file
+          scenario_outline.file
+        end  
+      end
+    end
+  end  
+end

data/lib/cucumber_testrail/testrail.rb

@@ -0,0 +1,108 @@
+#
+# TestRail API binding for Ruby (API v2, available since TestRail 3.0)
+#
+# Learn more:
+#
+# http://docs.gurock.com/testrail-api2/start
+# http://docs.gurock.com/testrail-api2/accessing
+#
+# Copyright Gurock Software GmbH
+#
+
+require 'net/http'
+require 'net/https'
+require 'uri'
+require 'json'
+
+module TestRail
+	class APIClient
+		@url = ''
+		@user = ''
+		@password = ''
+
+		attr_accessor :user
+		attr_accessor :password
+
+		def initialize(base_url,user,password)
+			if !base_url.match(/\/$/)
+				base_url += '/'
+			end
+			@url = base_url + 'index.php?/api/v2/'
+      @user ||=user
+      @password ||=password
+		end
+
+		#
+		# Send Get
+		#
+		# Issues a GET request (read) against the API and returns the result
+		# (as Ruby hash).
+		#
+		# Arguments:
+		#
+		# uri                 The API method to call including parameters
+		#                     (e.g. get_case/1)
+		#
+		def send_get(uri) 
+			_send_request('GET', uri, nil)
+		end
+
+		#
+		# Send POST
+		#
+		# Issues a POST request (write) against the API and returns the result
+		# (as Ruby hash).
+		#
+		# Arguments:
+		#
+		# uri                 The API method to call including parameters
+		#                     (e.g. add_case/1)
+		# data                The data to submit as part of the request (as
+		#                     Ruby hash, strings must be UTF-8 encoded)
+		#
+		def send_post(uri, data)
+			_send_request('POST', uri, data)
+		end
+
+		private
+		def _send_request(method, uri, data)
+			url = URI.parse(@url + uri)
+			if method == 'POST'
+				request = Net::HTTP::Post.new(url.path + '?' + url.query)
+				request.body = JSON.dump(data)
+			else
+				request = Net::HTTP::Get.new(url.path + '?' + url.query)
+			end
+			request.basic_auth(@user, @password)
+			request.add_field('Content-Type', 'application/json')
+
+			conn = Net::HTTP.new(url.host, url.port)
+			if url.scheme == 'https'
+				conn.use_ssl = true
+				conn.verify_mode = OpenSSL::SSL::VERIFY_NONE
+			end
+			response = conn.request(request)
+
+			if response.body && !response.body.empty?
+				result = JSON.parse(response.body)
+			else
+				result = {}
+			end
+
+			if response.code != '200'
+				if result && result.key?('error')
+					error = '"' + result['error'] + '"'
+				else
+					error = 'No additional error message received'
+				end
+				raise APIError.new('TestRail API returned HTTP %s (%s)' %
+					[response.code, error])
+			end
+			
+			result
+		end
+	end
+
+	class APIError < StandardError
+	end
+end

data/lib/cucumber_testrail/testrail_extensions.rb

@@ -0,0 +1,139 @@
+module Testrail 
+  module Scenario
+    def self.included(base)
+      base.extend(ClassMethods)
+      base.set_testrail_configuration(:status_map,{'passed'=>1,'failed'=>5,'undefined'=>7,'pending'=>6})
+    end
+
+    module ClassMethods
+
+      @@testrail_configuration={}
+
+      ##
+      # set a configuration property at the class level. Initially used to set the mapping between Cucumber test results and Testrail result ids
+      def set_testrail_configuration(property_name,property_value)
+        @@testrail_configuration[property_name] = property_value
+      end
+      ##
+      # get a named Testrail configuration property. Currenly only used for the results status map
+      def get_testrail_configuration(property_name)
+        @@testrail_configuration[property_name]
+      end
+
+    end
+
+    ##
+    # * Looks for a tagname at scenario, feature file and environment levels in that order
+    # * Unless the tagname == 'ignore_testrail' it will return the id of the tag. e.g. input tagname 'testcase' output '5'
+    # * If the tagname == 'ignore_testrail' then it will return nil or true depeneding if it can find the tag or the environment variable
+    def testrail_tag(tagname)
+      #check scenario, feature, environment in that order
+      result = nil
+      unless tagname =~ /ignore_testrail|manual|skip_result/
+        [source_tag_names,feature_tags.tags.map{|t| t.name}].each do | tags |
+          extracted_tag = tags.map{|tag| /^@#{tagname}_(\d+)$|^@#{tagname}_(.*)$/.match(tag)}.compact
+          if extracted_tag
+            if tagname=='jira'
+              result = extracted_tag.map{|e| e.to_a.compact[1]}.join(', ')
+            else  
+              result = extracted_tag.first.to_a.compact[1] # we compact it because there are two possible matches, if it hits the second then the first will be nil so after compacting 
+              # the result is always the first second element. Try it in irb to convince yourself
+            end
+            break
+          end  
+        end  
+      else
+        [source_tag_names,feature_tags.tags.map{|t| t.name}].each do | tags |
+          extracted_tag = tags.map{|tag| /#{tagname}/.match(tag)}.compact.first
+          if extracted_tag
+            result = true 
+            break
+          end  
+        end  
+      end
+      result ||=ENV[tagname.upcase]
+    end
+
+    ##
+    # returns true if the tag @ignore_testrail or environment variable IGNORE_TESTRAIL=true are found
+    # OR the suite and project id cannot be found
+    def ignore_testrail?
+      testrail_tag('ignore_testrail') || !suite || !project
+    end  
+    ##
+    # is this a manual scenario?
+    def is_manual?
+      testrail_tag('manual')
+    end
+    ##
+    # skip sending the result - useful for simply loading the testcases into Testrail
+    def skip_result?
+      testrail_tag('skip_result')
+    end
+    ##
+    # return the project id
+    def project
+      testrail_tag('project')
+    end  
+
+    ##
+    # return the suite id
+    def suite
+      testrail_tag('suite')
+    end  
+
+    ##
+    # return the sub_section id
+    def sub_section
+      testrail_tag('sub_section')
+    end  
+
+    ##
+    # return the testcase id
+    def testcase
+      testrail_tag('testcase')
+    end  
+
+    ##
+    # return the tesrun id
+    def testrun
+      testrail_tag('testrun')
+    end  
+
+    ##
+    # return test results if it failed, ready to write into the testcase test result
+    def test_result
+      if status =~ /failed/
+        "#{status} #{exception} line #{backtrace_line}"[0..249]
+      end  
+    end
+    ##
+    #return the jira ticket number if there is one
+    def jira
+      testrail_tag('jira')
+    end
+    ##
+    # translate the scenario status into a testrail id using the configured status map. This can be changed in your set up env.rb
+    # but by default it maps 
+    # * passed to 1
+    # * failed to 5
+    # * undefined to 2
+    # * pending to 2
+    def testrail_status
+      self.class.get_testrail_configuration(:status_map)[status.to_s] 
+    end
+
+    ##
+    # returns a hash of status_id and comment to write into Testrail test report
+    def testrail_test_report
+      {status_id:testrail_status,comment:test_result,defects:jira}
+    end
+
+    ##
+    # returns a hash of :title, :type_id and :custom_steps to create or update a Testrail testcase
+    # [TODO] parameterize the mapping of type_id to manual, in other installations it might be different
+    def testrail_testcase
+      {'title'=>title,'type_id'=>(is_manual? ? 7 : 1 ),'custom_steps'=>steps_as_string}
+    end
+  end
+end

data/lib/cucumber_testrail/version.rb

@@ -0,0 +1,3 @@
+module CucumberTestrail
+  VERSION = "1.3.0"
+end

data/report.log

@@ -0,0 +1,2 @@
+[webmock] Handling request: [get https://john.small%40bbc.com:314Sukra%21@paulokikiade.testrail.com/index.php?/api/v2/get_cases/1&suite_id=1] (disabled: false)
+[webmock] Identified request type (unhandled) for [get https://john.small%40bbc.com:314Sukra%21@paulokikiade.testrail.com/index.php?/api/v2/get_cases/1&suite_id=1]

data/spec/cucumber_ast_scenario_spec.rb

@@ -0,0 +1,15 @@
+require 'bundler'
+Bundler.require
+require "spec_helper"
+require 'rspec'
+require 'rspec/mocks'
+
+describe Cucumber::Ast::Scenario do
+#  let(:world)  do
+#    world  = class_double('Funky')
+#    world.extend subject 
+#    world
+#  end
+#
+end
+

data/spec/cucumber_testrail_spec.rb

@@ -0,0 +1,103 @@
+require 'bundler'
+Bundler.require
+require "spec_helper"
+require 'rspec'
+require 'rspec/mocks'
+describe CucumberTestrail do 
+  before(:each) do
+  ENV['TESTRAIL_BASE_URL']='https://paulokikiade.testrail.com' 
+  ENV['TESTRAIL_USER']    ='test.rail@bbc.com'
+  ENV['TESTRAIL_PASSWORD']='a_password'
+  end
+
+  let(:world)  do
+    world  = class_double('Funky')
+    world.extend subject 
+    world
+  end
+  
+  describe "#testrail" do
+   it "should raise an exception if any of the environment variables are not set up" do
+    ['TESTRAIL_BASE_URL','TESTRAIL_USER','TESTRAIL_PASSWORD'].each do | var |
+      save_env = ENV[var]
+      begin
+        ENV[var] = nil
+        expect{world.testrail}.to raise_exception(RuntimeError,/#{var}/)
+      ensure
+        ENV[var] = save_env
+      end
+    end  
+   end
+  end
+
+  describe "#send result" do 
+    it 'should not send the result if scenario.ignore_testrail is true' do
+      scenario = instance_double('Scenario',ignore_testrail?: true)
+      expect(world.send_result(scenario)).to be_falsey
+    end  
+  end
+
+  describe "#locate_a_test_case_by_name" do
+    it "doesn't find a phoney test" do
+      VCR.use_cassette('title_not_found') do
+        scenario = instance_double("Scenario",title:'A silly title',project:1,suite:1,sub_section:2)
+        expect(world.locate_a_test_case_by_name(scenario)).to be_nil
+      end
+    end
+
+    it "does find a real test" do
+      VCR.use_cassette('title_is_found') do
+        scenario = instance_double("Scenario",title:'Carousel - Test Case 1',project:1,suite:1,sub_section:2)
+        expect(world.locate_a_test_case_by_name(scenario)).not_to be_nil
+      end
+    end
+  end
+
+  describe '#create_a_test_case_from_scenario' do
+    let(:testrail_testcase) {{'title'=>'A test scenario','type_id'=>1,'custom_steps'=>'Given some steps'}}
+    let(:scenario) {instance_double("Scenario",suite:2,sub_section:2,testrail_testcase:testrail_testcase)}
+    it "should get a testcase id" do
+      VCR.use_cassette('create_a_test_case') do
+        expect(world.create_a_test_case_from_scenario(scenario)['id']).not_to be_nil
+      end
+    end
+
+    it "should put the steps into custom steps" do
+      VCR.use_cassette('create_a_test_case') do
+        expect(world.create_a_test_case_from_scenario(scenario)['custom_steps']).to eq('Given some steps')
+      end
+    end
+    it "should add the test case to the test suite" do
+      VCR.use_cassette('create_a_test_case') do
+        expect(world.create_a_test_case_from_scenario(scenario)['section_id']).to eq(2)
+      end
+    end
+    it "should set the testcase title" do
+      VCR.use_cassette('create_a_test_case') do
+        expect(world.create_a_test_case_from_scenario(scenario)['title']).to eq('A test scenario')
+      end
+    end  
+  end  
+
+  describe '#send_test_result_to_testrail' do 
+    let(:testrail_test_report) {   {status_id:1,defects:'none',comment:'comment'}}
+    let(:scenario) {instance_double("Scenario",testcase:10,testrun:5,testrail_test_report:testrail_test_report)}
+    it 'should return a valid test result' do 
+      VCR.use_cassette('create_a_test_result') do
+        expect(world.send_test_result_to_testrail(scenario)['id']).not_to be_nil
+      end
+    end
+    
+    it 'should set the defects' do 
+      VCR.use_cassette('create_a_test_result') do
+        expect(world.send_test_result_to_testrail(scenario)['defects']).to eq('none')
+      end
+    end
+
+    it 'should set the comment' do 
+      VCR.use_cassette('create_a_test_result') do
+        expect(world.send_test_result_to_testrail(scenario)['comment']).to eq('comment')
+      end
+    end
+  end
+end

data/spec/fixtures/vcr_cassettes/create_a_test_case.yml

@@ -0,0 +1,39 @@
+---
+http_interactions:
+- request:
+    method: post
+    uri: https://test.rail%40bbc.com:a_password@paulokikiade.testrail.com/index.php?/api/v2/add_case/2
+    body:
+      encoding: UTF-8
+      string: '{"title":"A test scenario","type_id":1,"custom_steps":"Given some steps"}'
+    headers:
+      Accept-Encoding:
+      - gzip;q=1.0,deflate;q=0.6,identity;q=0.3
+      Accept:
+      - "*/*"
+      User-Agent:
+      - Ruby
+      Content-Type:
+      - application/json
+  response:
+    status:
+      code: 200
+      message: OK
+    headers:
+      Date:
+      - Mon, 07 Jul 2014 09:51:06 GMT
+      Server:
+      - Apache
+      Content-Length:
+      - '322'
+      Content-Type:
+      - application/json; charset=utf-8
+      Set-Cookie:
+      - gr_sid=2f527; path=/
+    body:
+      encoding: UTF-8
+      string: '{"id":13,"title":"A test scenario","section_id":2,"type_id":1,"priority_id":4,"milestone_id":null,"refs":null,"created_by":2,"created_on":1404726666,"updated_by":2,"updated_on":1404726666,"estimate":null,"estimate_forecast":null,"suite_id":2,"custom_preconds":null,"custom_steps":"Given
+        some steps","custom_expected":null}'
+    http_version: 
+  recorded_at: Mon, 07 Jul 2014 09:51:11 GMT
+recorded_with: VCR 2.9.2

data/spec/fixtures/vcr_cassettes/create_a_test_result.yml

@@ -0,0 +1,38 @@
+---
+http_interactions:
+- request:
+    method: post
+    uri: https://test.rail%40bbc.com:a_password@paulokikiade.testrail.com/index.php?/api/v2/add_result_for_case/5/10
+    body:
+      encoding: UTF-8
+      string: '{"status_id":1,"defects":"none","comment":"comment"}'
+    headers:
+      Accept-Encoding:
+      - gzip;q=1.0,deflate;q=0.6,identity;q=0.3
+      Accept:
+      - "*/*"
+      User-Agent:
+      - Ruby
+      Content-Type:
+      - application/json
+  response:
+    status:
+      code: 200
+      message: OK
+    headers:
+      Date:
+      - Mon, 07 Jul 2014 12:35:24 GMT
+      Server:
+      - Apache
+      Content-Length:
+      - '163'
+      Content-Type:
+      - application/json; charset=utf-8
+      Set-Cookie:
+      - gr_sid=2f527; path=/
+    body:
+      encoding: UTF-8
+      string: '{"id":51,"test_id":25,"status_id":1,"created_by":2,"created_on":1404736525,"assignedto_id":null,"comment":"comment","version":null,"elapsed":null,"defects":"none"}'
+    http_version: 
+  recorded_at: Mon, 07 Jul 2014 12:35:29 GMT
+recorded_with: VCR 2.9.2

data/spec/fixtures/vcr_cassettes/title_is_found.yml

@@ -0,0 +1,45 @@
+---
+http_interactions:
+- request:
+    method: get
+    uri: https://test.rail%40bbc.com:a_password@paulokikiade.testrail.com/index.php?/api/v2/get_cases/1&section_id=2&suite_id=1
+    body:
+      encoding: US-ASCII
+      string: ''
+    headers:
+      Accept-Encoding:
+      - gzip;q=1.0,deflate;q=0.6,identity;q=0.3
+      Accept:
+      - "*/*"
+      User-Agent:
+      - Ruby
+      Content-Type:
+      - application/json
+  response:
+    status:
+      code: 200
+      message: OK
+    headers:
+      Date:
+      - Mon, 07 Jul 2014 09:26:34 GMT
+      Server:
+      - Apache
+      Content-Length:
+      - '1057'
+      Content-Type:
+      - application/json; charset=utf-8
+      Cache-Control:
+      - private
+      Set-Cookie:
+      - gr_sid=b7690; path=/
+    body:
+      encoding: UTF-8
+      string: '[{"id":1,"title":"Carousel - Test Case 1","section_id":1,"type_id":2,"priority_id":5,"milestone_id":null,"refs":null,"created_by":1,"created_on":1403122853,"updated_by":1,"updated_on":1403122853,"estimate":"2m","estimate_forecast":"2m","suite_id":1,"custom_preconds":"Ensure
+        hope page is present","custom_steps":"test homepage renders in 1 sec","custom_expected":"Homepage
+        renders in 1 sec"},{"id":2,"title":"Ensure test is tested","section_id":1,"type_id":6,"priority_id":4,"milestone_id":null,"refs":null,"created_by":1,"created_on":1403122907,"updated_by":1,"updated_on":1403122907,"estimate":null,"estimate_forecast":null,"suite_id":1,"custom_preconds":"test
+        test test","custom_steps":"test test test test","custom_expected":"test test"},{"id":3,"title":"3rd
+        test","section_id":1,"type_id":6,"priority_id":4,"milestone_id":null,"refs":null,"created_by":1,"created_on":1403122925,"updated_by":1,"updated_on":1403122925,"estimate":null,"estimate_forecast":null,"suite_id":1,"custom_preconds":"test
+        ","custom_steps":"test ","custom_expected":"test again"}]'
+    http_version: 
+  recorded_at: Mon, 07 Jul 2014 09:26:39 GMT
+recorded_with: VCR 2.9.2

data/spec/fixtures/vcr_cassettes/title_not_found.yml

@@ -0,0 +1,45 @@
+---
+http_interactions:
+- request:
+    method: get
+    uri: https://test.rail%40bbc.com:a_password@paulokikiade.testrail.com/index.php?/api/v2/get_cases/1&section_id=2&suite_id=1
+    body:
+      encoding: US-ASCII
+      string: ''
+    headers:
+      Accept-Encoding:
+      - gzip;q=1.0,deflate;q=0.6,identity;q=0.3
+      Accept:
+      - "*/*"
+      User-Agent:
+      - Ruby
+      Content-Type:
+      - application/json
+  response:
+    status:
+      code: 200
+      message: OK
+    headers:
+      Date:
+      - Mon, 07 Jul 2014 09:21:56 GMT
+      Server:
+      - Apache/2.2.14 (Ubuntu)
+      Content-Length:
+      - '1057'
+      Content-Type:
+      - application/json; charset=utf-8
+      Cache-Control:
+      - private
+      Set-Cookie:
+      - gr_sid=dba88; path=/
+    body:
+      encoding: UTF-8
+      string: '[{"id":1,"title":"Carousel - Test Case 1","section_id":1,"type_id":2,"priority_id":5,"milestone_id":null,"refs":null,"created_by":1,"created_on":1403122853,"updated_by":1,"updated_on":1403122853,"estimate":"2m","estimate_forecast":"2m","suite_id":1,"custom_preconds":"Ensure
+        hope page is present","custom_steps":"test homepage renders in 1 sec","custom_expected":"Homepage
+        renders in 1 sec"},{"id":2,"title":"Ensure test is tested","section_id":1,"type_id":6,"priority_id":4,"milestone_id":null,"refs":null,"created_by":1,"created_on":1403122907,"updated_by":1,"updated_on":1403122907,"estimate":null,"estimate_forecast":null,"suite_id":1,"custom_preconds":"test
+        test test","custom_steps":"test test test test","custom_expected":"test test"},{"id":3,"title":"3rd
+        test","section_id":1,"type_id":6,"priority_id":4,"milestone_id":null,"refs":null,"created_by":1,"created_on":1403122925,"updated_by":1,"updated_on":1403122925,"estimate":null,"estimate_forecast":null,"suite_id":1,"custom_preconds":"test
+        ","custom_steps":"test ","custom_expected":"test again"}]'
+    http_version: 
+  recorded_at: Mon, 07 Jul 2014 09:22:01 GMT
+recorded_with: VCR 2.9.2

data/spec/spec_helper.rb

@@ -0,0 +1,11 @@
+require 'bundler'
+Bundler.require
+require "cucumber_testrail"
+require 'vcr'
+require 'rspec/mocks'
+
+VCR.configure do |c|
+  c.cassette_library_dir = 'spec/fixtures/vcr_cassettes'
+  c.hook_into :webmock # or :fakeweb
+  #c.debug_logger = File.open('report.log', 'w')
+end

data/tasks/rspec.rake

@@ -0,0 +1,4 @@
+
+require 'rspec/core/rake_task'
+
+RSpec::Core::RakeTask.new(:spec)

metadata

@@ -0,0 +1,188 @@
+--- !ruby/object:Gem::Specification
+name: cucumber_testrail
+version: !ruby/object:Gem::Version
+  version: 1.3.0
+platform: ruby
+authors:
+- John Small
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2016-03-31 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: cucumber
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.3'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.3'
+- !ruby/object:Gem::Dependency
+  name: bundler
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.6'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.6'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.3'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.3'
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+- !ruby/object:Gem::Dependency
+  name: vcr
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.9'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.9'
+- !ruby/object:Gem::Dependency
+  name: webmock
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.18'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.18'
+- !ruby/object:Gem::Dependency
+  name: rdoc
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '4.1'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '4.1'
+- !ruby/object:Gem::Dependency
+  name: geminabox
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.12'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.12'
+description: |-
+  Call cucumber_testrail.send_result(scenario) in an After hook. It will match existing testcases by scenario title or add new testcases
+    and then add the result of the run to TestRail. See the README.md
+email:
+- john.small@bbc.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- ".gitignore"
+- ".ruby-gemset"
+- ".ruby-version"
+- Gemfile
+- LICENSE.txt
+- README.md
+- Rakefile
+- cucumber_testrail.gemspec
+- lib/cucumber_testrail.rb
+- lib/cucumber_testrail/external_reporting.rb
+- lib/cucumber_testrail/reopen_scenario.rb
+- lib/cucumber_testrail/testrail.rb
+- lib/cucumber_testrail/testrail_extensions.rb
+- lib/cucumber_testrail/version.rb
+- report.log
+- spec/cucumber_ast_scenario_spec.rb
+- spec/cucumber_testrail_spec.rb
+- spec/fixtures/vcr_cassettes/create_a_test_case.yml
+- spec/fixtures/vcr_cassettes/create_a_test_result.yml
+- spec/fixtures/vcr_cassettes/title_is_found.yml
+- spec/fixtures/vcr_cassettes/title_not_found.yml
+- spec/spec_helper.rb
+- tasks/rspec.rake
+homepage: 
+licenses:
+- MIT
+metadata: {}
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 2.2.2
+signing_key: 
+specification_version: 4
+summary: Sync Cucumber specs and results to TestRail
+test_files:
+- spec/cucumber_ast_scenario_spec.rb
+- spec/cucumber_testrail_spec.rb
+- spec/fixtures/vcr_cassettes/create_a_test_case.yml
+- spec/fixtures/vcr_cassettes/create_a_test_result.yml
+- spec/fixtures/vcr_cassettes/title_is_found.yml
+- spec/fixtures/vcr_cassettes/title_not_found.yml
+- spec/spec_helper.rb