res 1.0.0

Sign up to get free protection for your applications and to get access to all the features.
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: 127358ee22371eb6201add12abb4a4aec0534ef7
4
+ data.tar.gz: 7e4bdc7f8c10691fe03569bf3a8762cde6aaada9
5
+ SHA512:
6
+ metadata.gz: 934947a0fb56156732515d456b454819146b83ce8e3b7346a2bba55faafe47b54b716093a22aaf2201a3dda3f5fdf1cd7010ad812fab92c30e16a70dc55d89e4
7
+ data.tar.gz: 0507cb3031cb9301ebc6281d4b45ce9316ad3877b1a4ea53d69351bbaf121abf3674691d5f6d039e8299e9f55bce40799d9baf74038476b77d314c8870063fe3
@@ -0,0 +1,96 @@
1
+ # Res
2
+
3
+ Res is a collection of formatting tools for making test
4
+ result reporting easier and more consistent. The concept is that
5
+ rather than writing formatters over and over again for different testing
6
+ tools and result repositories, we standardise on one format for test
7
+ results (an Intermediate Representation) and write a single formatter
8
+ and submitter for each runner and repository.
9
+
10
+ Example: Reporting a testing tool output to testrail
11
+
12
+ Formatter ---------> Intermediate --------> Reporter
13
+ or Parser Representation (TestRail)
14
+
15
+ ## Frontends
16
+
17
+ There are three different types of front end:
18
+
19
+ 1. Formatters -- libraries that can be loaded into a test runner to generate IR output
20
+ 2. Parsers -- scripts that parse result files into IR after a test runner has completed
21
+ 3. Native -- custom test runners that use IR as their native format
22
+
23
+ IR is a json formatted file that captures a hierarchical definition of
24
+ a test suite alongside results and metadata.
25
+
26
+ ## Formatters and Parsers
27
+ You can dump a Res IR results file using a cucumber formatter or parse xunit output into res format.
28
+
29
+ ## Cucumber
30
+
31
+ cucumber -f pretty -f Res::Formatters::RubyCucumber -o './cucumber.res'
32
+ Note: This cucumber formatter works for cucumber version < 2.0
33
+
34
+ ## Junit
35
+
36
+ ./bin/res.rb --junit '/path/to/xunit_result.xml'
37
+ Note: The Res output of the xunit parser is saved in the current directory
38
+
39
+ ## Reporters
40
+
41
+ There are a number of backend reporters, for taking IR and producing a report from
42
+ it, or submitting the IR results to a test repository or test management
43
+ tool.
44
+
45
+ If you have a Res IR file, you can submit using a reporter:
46
+
47
+ ./bin/res.rb --res '/path/to/file.res' --submit REPORTER [... options]
48
+
49
+ ### Hive
50
+
51
+ Hive CI uses a Res reporter for result submission, the api arguments look like this:
52
+
53
+ Res.submit_results(
54
+ :reporter => :hive,
55
+ :ir => 'path/to/file.res',
56
+ :url => 'http://hive.local',
57
+ :job_id => 10723
58
+ )
59
+
60
+ ### TestRail
61
+
62
+ ./bin/res.rb --res '/path/to/file.res' --submit testrail --config-file '/path/to/.test_rail.yaml'
63
+
64
+ Our TestRail reporter currently be used to sync a suite with TestRail, and
65
+ to submit test results against a test run. You will need to create a
66
+ config file in your project in order for our reporter to know where to sync
67
+ the tests and put the results.
68
+
69
+ Your config file should be called .test_rail.yaml, and should look like this:
70
+
71
+ namespace: 'NameSpace'
72
+ user: 'user@example.com'
73
+ password: 'passw0rd'
74
+ project: 'MyProjectName'
75
+ suite: 'MySuite'
76
+ run_name: 'RunName' or run_id: '1234'
77
+
78
+ ### Testmine
79
+
80
+ The Testmine reporter is very similar to the TestRail reporter, but doesn't
81
+ require you to sync your test definitions before you submit a run.
82
+
83
+ You need to create a project file in the root of your test code, the file
84
+ should be called .testmine.yaml, and should look like this:
85
+
86
+ testmine_url: 'mytestmineinstance.bbc.co.uk'
87
+ authentication: 'authenticationkey'
88
+ project: 'MyProjectName'
89
+ component: 'Android acceptance'
90
+ suite: 'MySuiteName'
91
+
92
+ ## License
93
+
94
+ *Res* is available to everyone under the terms of the MIT open source licence. Take a look at the LICENSE file in the code.
95
+
96
+ Copyright (c) 2015 BBC
@@ -0,0 +1,45 @@
1
+ require 'res/ir'
2
+
3
+ # Res API
4
+ module Res
5
+
6
+ # Report Res IR to a test repository or similar
7
+ def self.submit_results(args)
8
+ reporter_class = Res.reporter_class(args[:reporter])
9
+ reporter = reporter_class.new( args )
10
+
11
+ ir = Res::IR.load(args[:ir])
12
+
13
+ reporter.submit_results( ir, args )
14
+ end
15
+
16
+ def self.reporter_class(type)
17
+ case type
18
+ when :test_rail
19
+ require 'res/reporters/test_rail'
20
+ Res::Reporters::TestRail
21
+ when :hive
22
+ require 'res/reporters/hive'
23
+ Res::Reporters::Hive
24
+ when :testmine
25
+ require 'res/reporters/testmine'
26
+ Res::Reporters::Testmine
27
+ end
28
+ end
29
+
30
+ def self.parse_results(args)
31
+ parser_class = Res.parser_class(args[:parser])
32
+ parser_class.new(args[:file])
33
+ end
34
+
35
+ def self.parser_class(type)
36
+ case type
37
+ when :junit
38
+ require 'res/parsers/junit'
39
+ Res::Parsers::Junit
40
+ else
41
+ raise "#{type} parser not Implemented"
42
+ end
43
+ end
44
+
45
+ end
@@ -0,0 +1,59 @@
1
+ require 'yaml'
2
+
3
+
4
+ module Res
5
+ class Config
6
+ attr_accessor :struct, :required, :optional, :prepend
7
+
8
+ def initialize(items, args = {})
9
+ @required = *items
10
+ @optional = args[:optional] || []
11
+ @prepend = args[:pre_env] || ''
12
+ items = required + optional
13
+ @struct = Struct.new(*items).new
14
+ end
15
+
16
+ # Load in config -- this can come from three places:
17
+ # 1. Arguments passed to the initializer
18
+ # 2. From environment variables
19
+ # 3. From a config file
20
+ def process( args = {} )
21
+
22
+ config_from_file = {}
23
+ if args[:config_file]
24
+ config_from_file = load_from_config(args[:config_file])
25
+ args.delete(:config_file)
26
+ end
27
+
28
+ missing = []
29
+ struct.members.each do |item|
30
+ struct[item] = args[item] || ENV[(prepend + item.to_s).upcase] || config_from_file[item] || nil
31
+ missing << item if (struct[item].nil? && required.include?(item))
32
+ end
33
+ raise "Missing configuration: " + missing.join( ", ") if missing.any?
34
+ end
35
+
36
+ def load_from_config(config_file)
37
+ config = {}
38
+ if File.exists?(config_file)
39
+ config = Res::Config.symbolize_keys(YAML::load(File.open(config_file)))
40
+ else
41
+ raise "Couldn't find config file '#{config_file}'"
42
+ end
43
+ config
44
+ end
45
+
46
+ def self.symbolize_keys(hash)
47
+ symbolized_hash = {}
48
+ hash.each do |key, value|
49
+ symbolized_hash[key.to_sym] = value
50
+ end
51
+ symbolized_hash
52
+ end
53
+
54
+ def method_missing(m, *args, &block)
55
+ struct.send(m,*args, &block)
56
+ end
57
+
58
+ end
59
+ end
@@ -0,0 +1,284 @@
1
+ # Formatter for ruby cucumber
2
+
3
+ require 'fileutils'
4
+ require 'res/ir'
5
+ require 'cucumber/formatter/io'
6
+
7
+ module Res
8
+ module Formatters
9
+ class RubyCucumber
10
+ include FileUtils
11
+ include ::Cucumber::Formatter::Io
12
+
13
+ def initialize(runtime, path_or_io, options)
14
+ cucumber_version = %x(cucumber --version)
15
+ @cucumber_version = cucumber_version.gsub("\n","")
16
+
17
+ @runtime = runtime
18
+ @io = ensure_io(path_or_io, "reporter")
19
+ @options = options
20
+ @exceptions = []
21
+ @indent = 0
22
+ @prefixes = options[:prefixes] || {}
23
+ @delayed_messages = []
24
+ @_start_time = Time.now
25
+ end
26
+
27
+ def before_features(features)
28
+ @_features = []
29
+ end
30
+
31
+ # Once everything has run -- whack it in a ResultIR object and
32
+ # dump it as json
33
+ def after_features(features)
34
+ results = @_features
35
+ ir = ::Res::IR.new( :started => @_start_time,
36
+ :finished => Time.now(),
37
+ :results => results,
38
+ :type => 'Cucumber' )
39
+ @io.puts ir.json
40
+ end
41
+
42
+ def before_feature(feature)
43
+ @_feature = {}
44
+ @_context = {}
45
+ @_feature[:started] = Time.now()
46
+ begin
47
+ if @cucumber_version.to_f < 1.3.to_f
48
+ uri = feature.file.to_s
49
+ else
50
+ uri = feature.location.to_s
51
+ end
52
+
53
+ hash = RubyCucumber.split_uri( uri )
54
+ @_feature[:file] = hash[:file]
55
+ @_feature[:line] = hash[:line]
56
+ @_feature[:urn] = hash[:urn]
57
+ rescue
58
+ @_feature[:uri] = 'unknown'
59
+ end
60
+ @_features << @_feature
61
+ @_context = @_feature
62
+ end
63
+
64
+ def comment_line(comment_line)
65
+ @_context[:comments] = [] if !@_context[:comments]
66
+ @_context[:comments] << comment_line
67
+ end
68
+
69
+ def after_tags(tags)
70
+ end
71
+
72
+ def tag_name(tag_name)
73
+ @_context[:tags] = [] if !@_context[:tag]
74
+ # Strip @ from tags
75
+ @_context[:tags] << tag_name[1..-1]
76
+ end
77
+
78
+ # { :type => 'Feature',
79
+ # :name => 'Feature name',
80
+ # :description => "As a blah\nAs a blah\n" }
81
+ def feature_name(keyword, name)
82
+ @_feature[:type] = "Cucumber::" + keyword.gsub(/\s+/, "")
83
+
84
+ lines = name.split("\n")
85
+ lines = lines.collect { |l| l.strip }
86
+
87
+ @_feature[:name] = lines.shift
88
+ @_feature[:description] = lines.join("\n")
89
+ end
90
+
91
+ def after_feature(feature)
92
+ @_feature[:finished] = Time.now()
93
+ end
94
+
95
+ def before_feature_element(feature_element)
96
+
97
+ @_feature_element = {}
98
+ @_context = {}
99
+ @_feature_element[:started] = Time.now
100
+ begin
101
+ if @cucumber_version.to_f < 1.3.to_f
102
+ uri = feature_element.file_colon_line
103
+ else
104
+ uri = feature_element.location.to_s
105
+ end
106
+ hash = RubyCucumber.split_uri( uri )
107
+ @_feature_element[:file] = hash[:file]
108
+ @_feature_element[:line] = hash[:line]
109
+ @_feature_element[:urn] = hash[:urn]
110
+ rescue => e
111
+ @_feature_element[:error] = e.message
112
+ @_feature_element[:file] = 'unknown'
113
+ end
114
+
115
+ @_feature[:children] = [] if ! @_feature[:children]
116
+
117
+ @_feature[:children] << @_feature_element
118
+ @_context = @_feature_element
119
+ end
120
+
121
+ # After a scenario
122
+ def after_feature_element(feature_element)
123
+ @_context = {}
124
+
125
+ if feature_element.respond_to? :status
126
+ @_feature_element[:status] = feature_element.status
127
+ end
128
+ @_feature_element[:finished] = Time.now
129
+ end
130
+
131
+ def before_background(background)
132
+ #@_context[:background] = background
133
+ end
134
+
135
+ def after_background(background)
136
+ end
137
+
138
+ def background_name(keyword, name, file_colon_line, source_indent)
139
+ end
140
+
141
+ # def before_examples_array(examples_array)
142
+ # @indent = 4
143
+ # @io.puts
144
+ # @visiting_first_example_name = true
145
+ # end
146
+ #
147
+
148
+ def examples_name(keyword, name)
149
+ # @io.puts unless @visiting_first_example_name
150
+ # @visiting_first_example_name = false
151
+ # names = name.strip.empty? ? [name.strip] : name.split("\n")
152
+ # @io.puts(" #{keyword}: #{names[0]}")
153
+ # names[1..-1].each {|s| @io.puts " #{s}" } unless names.empty?
154
+ # @io.flush
155
+ # @indent = 6
156
+ # @scenario_indent = 6
157
+ end
158
+ #
159
+
160
+ def scenario_name(keyword, name, file_colon_line, source_indent)
161
+ @_context[:type] = "Cucumber::" + keyword.gsub(/\s+/, "")
162
+ @_context[:name] = name || ''
163
+ end
164
+
165
+ def before_step(step)
166
+ @_step = {}
167
+
168
+ # Background steps can appear totally divorced from scenerios (feature
169
+ # elements). Need to make sure we're not including them as children
170
+ # to scenario that don't exist
171
+ return if @_feature_element && @_feature_element[:finished]
172
+
173
+ @_feature_element = {} if !@_feature_element
174
+ @_feature_element[:children] = [] if !@_feature_element[:children]
175
+ @_feature_element[:children] << @_step
176
+ @_context = @_step
177
+ end
178
+
179
+ # def before_step_result(keyword, step_match, multiline_arg, status, exception, source_indent, background, file_colon_line)
180
+ # @hide_this_step = false
181
+ # if exception
182
+ # if @exceptions.include?(exception)
183
+ # @hide_this_step = true
184
+ # return
185
+ # end
186
+ # @exceptions << exception
187
+ # end
188
+ # if status != :failed && @in_background ^ background
189
+ # @hide_this_step = true
190
+ # return
191
+ # end
192
+ # @status = status
193
+ # end
194
+
195
+ # Argument list changed after cucumber 1.4, hence the *args
196
+ def step_name(keyword, step_match, status, source_indent, background, *args)
197
+
198
+ file_colon_line = args[0] if args[0]
199
+
200
+ @_step[:type] = "Cucumber::Step"
201
+ name = keyword + step_match.format_args(lambda{|param| %{#{param}}})
202
+ @_step[:name] = name
203
+ @_step[:status] = status
204
+ #@_step[:background] = background
205
+ @_step[:type] = "Cucumber::Step"
206
+
207
+ end
208
+
209
+ # def doc_string(string)
210
+ # return if @options[:no_multiline] || @hide_this_step
211
+ # s = %{"""\n#{string}\n"""}.indent(@indent)
212
+ # s = s.split("\n").map{|l| l =~ /^\s+$/ ? '' : l}.join("\n")
213
+ # @io.puts(format_string(s, @current_step.status))
214
+ # @io.flush
215
+ # end
216
+ #
217
+
218
+ def exception(exception, status)
219
+ @_context[:message] = exception.to_s
220
+ end
221
+
222
+ def before_multiline_arg(multiline_arg)
223
+ # return if @options[:no_multiline] || @hide_this_step
224
+ # @table = multiline_arg
225
+ end
226
+
227
+ def after_multiline_arg(multiline_arg)
228
+ @_context[:args] = multiline_arg.to_s.gsub(/\e\[(\d+)m/, '')
229
+ @_table = nil
230
+ end
231
+
232
+ # Before a scenario outline is encountered
233
+ def before_outline_table(outline_table)
234
+ # Scenario outlines appear as children like normal scenarios,
235
+ # but really we just want to construct normal-looking children
236
+ # from them
237
+ @_outlines = @_feature_element[:children]
238
+ @_table = []
239
+ end
240
+
241
+ def after_outline_table(outline_table)
242
+ headings = @_table.shift
243
+ description = @_outlines.collect{ |o| o[:name] }.join("\n") + "\n" + headings[:name]
244
+ @_feature_element[:children] = @_table
245
+ @_feature_element[:description] = description
246
+ end
247
+
248
+ def before_table_row(table_row)
249
+ @_current_table_row = { :type => 'Cucumber::ScenarioOutline::Example' }
250
+ @_table = [] if !@_table
251
+ end
252
+
253
+ def after_table_row(table_row)
254
+ if table_row.class == Cucumber::Ast::OutlineTable::ExampleRow
255
+ @_current_table_row[:name] = table_row.name
256
+ if table_row.exception
257
+ @_current_table_row[:message] = table_row.exception.to_s
258
+ end
259
+ if table_row.scenario_outline
260
+ @_current_table_row[:status] = table_row.status
261
+ end
262
+ @_current_table_row[:line] = table_row.line
263
+ @_current_table_row[:urn] = @_feature_element[:file] + ":" + table_row.line.to_s
264
+ @_table << @_current_table_row
265
+ end
266
+ end
267
+
268
+ def after_table_cell(cell)
269
+ end
270
+
271
+ def table_cell_value(value, status)
272
+ @_current_table_row[:children] = [] if !@_current_table_row[:children]
273
+ @_current_table_row[:children] << { :type => "Cucumber::ScenarioOutline::Parameter",
274
+ :name => value, :status => status }
275
+ end
276
+
277
+ def self.split_uri(uri)
278
+ strings = uri.rpartition(/:/)
279
+ { :file => strings[0], :line => strings[2].to_i, :urn => uri }
280
+ end
281
+
282
+ end
283
+ end
284
+ end
@@ -0,0 +1,81 @@
1
+ require 'json'
2
+
3
+ module Res
4
+ class IR
5
+ attr_accessor :hash, :results, :type, :start_time, :end_time, :world
6
+ attr_accessor :project, :suite, :target, :hive_job_id
7
+
8
+ def self.load(file)
9
+ f = File.open file
10
+ hash = JSON.load(f, nil, :symbolize_names => true )
11
+ Res::IR.new( hash )
12
+ end
13
+
14
+ # Expects hash of:
15
+ # :results => { }
16
+ # :type => test_runner
17
+ # :start_time => Time the tests started
18
+ # :end_time => Time they completed
19
+ def initialize( options = {} )
20
+ @results = options[:results] or raise "No results data"
21
+ @type = options[:type] or raise "No type provided (e.g. 'Cucumber')"
22
+ @started = options[:started] or raise "Need to provide a start time"
23
+ @finished = options[:finished] or raise "Need to provide an end time"
24
+ end
25
+
26
+ # Dump as json
27
+ def json
28
+ hash = {
29
+ :started => @started,
30
+ :finished => @finished,
31
+ :results => @results,
32
+ :type => @type }
33
+
34
+ # Merge in the world information if it's available
35
+ hash[:world] = world if world
36
+ hash[:hive_job_id] = hive_job_id if hive_job_id
37
+
38
+ JSON.pretty_generate( hash )
39
+ end
40
+
41
+ # Pluck out the actual test nodes from the contexts
42
+ def tests
43
+ IR.find_tests(results).flatten
44
+ end
45
+
46
+ def count(status)
47
+ tests.count { |t| t[:status].to_sym == status.to_sym }
48
+ end
49
+
50
+ # Returns a simple array of test information
51
+ # [ { :name => 'test1', :urn => 'file/tests.t:32', :status => 'passed', :time => 12.04 },
52
+ # { :name => 'test2', :urn => 'file/tests.t:36', :status => 'failed', :time => } ]
53
+ def flat_format
54
+ self.tests.collect do |t|
55
+ { :name => t[:name],
56
+ :urn => t[:urn],
57
+ :status => t[:status] }
58
+ end
59
+ end
60
+
61
+ private
62
+
63
+ # Recursive function for retrieving test nodes
64
+ def self.find_tests(nodes)
65
+ tests = []
66
+ nodes.each do |n|
67
+ if IR.is_a_test?(n)
68
+ tests << n
69
+ elsif n[:children]
70
+ tests << IR.find_tests(n[:children])
71
+ end
72
+ end
73
+ tests
74
+ end
75
+
76
+ def self.is_a_test?(node)
77
+ !node[:status].nil?
78
+ end
79
+
80
+ end
81
+ end
@@ -0,0 +1,23 @@
1
+ module Res
2
+ class Mappings
3
+
4
+ attr_accessor :context, :case, :type
5
+
6
+ def initialize(type)
7
+ @type = type
8
+ set_type
9
+ end
10
+
11
+ def set_type
12
+ case @type
13
+ when "Junit"
14
+ @context = ["JUnit::testsuite", "JUnit::testsuites"]
15
+ @case = ["JUnit::testcase"]
16
+ when "Cucumber"
17
+ @context = ["Cucumber::Feature", "Cucumber::ScenarioOutline"]
18
+ @case = ["Cucumber::Scenario"]
19
+ end
20
+ end
21
+
22
+ end
23
+ end
@@ -0,0 +1,65 @@
1
+ require 'ox'
2
+ require 'json'
3
+ require 'res/ir'
4
+
5
+ module Res
6
+ module Parsers
7
+ class Junit
8
+ attr_accessor :io
9
+
10
+ def initialize(junit_xml)
11
+ file = File.open(junit_xml, "rb")
12
+ begin
13
+ junit = Ox.parse(file.read)
14
+ rescue Ox::ParseError => e
15
+ raise "Invalid xunit XML format. Error: #{e}"
16
+ end
17
+ file.close
18
+ result = Array.new
19
+ result = attach_suite(junit)
20
+ ir = ::Res::IR.new( :type => 'Junit',
21
+ :started => "",
22
+ :finished => Time.now(),
23
+ :results => result
24
+ )
25
+ @io = File.open("./junit.res", "w")
26
+ @io.puts ir.json
27
+ @io.close
28
+ end
29
+
30
+ def attach_cases(node)
31
+ testcase = Hash.new
32
+ testcase["type"] = "JUnit::" + node.value
33
+ testcase["name"] = node.attributes[:name]
34
+ testcase["classname"] = node.attributes[:classname] if testcase["classname"] != nil
35
+ testcase["duration"] = node.attributes[:time]
36
+ testcase["status"] = "passed"
37
+ if node.nodes[0] != nil
38
+ testcase["status"] = "failed" if node.nodes[0].value == "failure" or node.nodes[0].value == "error"
39
+ testcase["status"] = "notrun" if node.nodes[0].value == "skipped"
40
+ end
41
+ testcase
42
+ end
43
+
44
+ def attach_suite(component)
45
+ suite = Array.new
46
+ index = 0
47
+ component.nodes.each do |node|
48
+ if node.value == "testcase"
49
+ suite[index] = Hash.new
50
+ suite[index] = attach_cases(node)
51
+ else
52
+ suite[index] = Hash.new
53
+ suite[index]["type"] = "JUnit::" + node.value
54
+ suite[index]["name"] = node.attributes[:name]
55
+ suite[index]["classname"] = node.attributes[:classname] if suite[index]["classname"] != nil
56
+ suite[index]["children"] = attach_suite(node)
57
+ end # if
58
+ index += 1
59
+ end # each
60
+ suite
61
+ end # def attach_suite
62
+ end # class JUnit
63
+ end # class Parsers
64
+ end # class Res
65
+
@@ -0,0 +1,42 @@
1
+ require 'hive/messages'
2
+ require 'json'
3
+
4
+ module Res
5
+ module Reporters
6
+ class Hive
7
+
8
+ def initialize( args = {} )
9
+
10
+ if args[:url]
11
+ ::Hive::Messages.configure do |config|
12
+ config.base_path = args[:url]
13
+ config.pem_file = args[:cert]
14
+ config.ssl_verify_mode = args[:ssl_verify_mode]
15
+ end
16
+ end
17
+
18
+ end
19
+
20
+ def submit_results(ir, args)
21
+
22
+ # Still include count summaries for backward compatability
23
+ params = { :failed_count => ir.count(:failed),
24
+ :passed_count => ir.count(:passed),
25
+ :errored_count => ir.count(:errored),
26
+ :running_count => ir.count(:running),
27
+ :result_details => ir.flat_format.to_json }
28
+
29
+ hive_job = ::Hive::Messages::Job.new(:job_id => args[:job_id])
30
+ begin
31
+ update_response = hive_job.update_results(params)
32
+ #log "Hive results update response: #{update_response}"
33
+ rescue
34
+ #log "Hive::Messages couldn't hit the hive, retrying once more"
35
+ update_response = hive_job.update_results(params)
36
+ #log "Hive results update response: #{update_response}"
37
+ end
38
+ end
39
+
40
+ end
41
+ end
42
+ end
@@ -0,0 +1,150 @@
1
+ require 'test_rail'
2
+ require 'res/ir'
3
+ require 'res/mappings'
4
+ require 'res/config'
5
+
6
+ module Res
7
+ module Reporters
8
+ class TestRail
9
+
10
+ attr_accessor :ir, :case_status, :config, :project, :suite
11
+ def initialize(args)
12
+ @url = args[:url]
13
+ @config = Res::Config.new([:user, :password, :namespace, :project, :suite], :optional => [:run_id, :run_name], :pre_env => 'test_rail')
14
+ config.process(args)
15
+
16
+ @case_status = {}
17
+ @io = File.new("./.test_rail_reporter.log","w+")
18
+
19
+ test_rail_project = config.project
20
+ @suite_name = config.suite
21
+
22
+ begin
23
+ @project = tr.find_project(:name => test_rail_project)
24
+ rescue
25
+ @io.puts "Project: #{test_rail_project} could not be found"
26
+ @io.puts " Please create a project first to contain test suite #{@suite_name}"
27
+ @io.close
28
+ return "Project #{test_rail_project} could not be found"
29
+ end
30
+ end # initialize
31
+
32
+ # Creates a new suite within testrail
33
+ def sync_tests(args)
34
+ @ir = args[:ir]
35
+ @mappings = Res::Mappings.new(@ir.type)
36
+
37
+ suite = @project.find_or_create_suite(:name => @suite_name, :id => @project.id)
38
+ @io.puts "Syncing Suite"
39
+
40
+ i = 0
41
+ while i < @ir.results.count
42
+ section = suite.find_or_create_section(:project_id => @project.id, :suite_id => suite.id, :name => @ir.results[i][:name])
43
+ create_suite(@ir.results[i], @project.id, suite, section)
44
+ i += 1
45
+ end # while
46
+ @synced = true
47
+ @io.puts "> Sync Successful"
48
+ end
49
+
50
+ # Submits run against suite
51
+ # Either creates a new run using run_name or use existing run_id
52
+ def submit_results(args = {})
53
+ sync_tests(args) if !@synced
54
+ suite = @project.find_suite(:name => @suite_name)
55
+
56
+ run_name = @config.run_name || args[:run_name] || nil
57
+ run_id = @config.run_id || args[:run_id] || nil
58
+
59
+ if !run_name.nil?
60
+ run_name = @config.run_name
61
+ @io.puts "> Created new run with name #{run_name}"
62
+ run_id = @tr.add_run( :project_id => @project.id, :name => run_name, :description => args[:run_description], :suite_id => suite.id ).id
63
+ @io.puts "> Created new run: #{run_id}"
64
+
65
+ elsif !run_id.nil?
66
+ run_id = @config.run_id
67
+ begin
68
+ run = @tr.get_run(:run_id => @config.run_id)
69
+ @io.puts "> Found run with id #{run_id}"
70
+ rescue
71
+ @io.puts "> Couldn't find run with id #{run_id}"
72
+ @io.close
73
+ return "Couldn't find run with id #{run_id}"
74
+ end
75
+
76
+ else
77
+ @io.puts "> run_name and run_id are either nil or not specified"
78
+ @io.close
79
+ return "run_name and run_id are either nil or not specified"
80
+
81
+ end
82
+
83
+ i = 0
84
+ if @case_status.empty?
85
+ while i < @ir.results.count
86
+ begin
87
+ section = suite.find_section(:name => @ir.results[i][:name])
88
+ rescue
89
+ @io.puts "> Couldn't find section with name #{@ir.results[i][:name]}"
90
+ puts "Couldn't find section with name #{@ir.results[i][:name]}"
91
+ end
92
+ case_details(@ir.results[i], section)
93
+ i += 1
94
+ end # while
95
+ end # ifa
96
+ add_case_status(run_id)
97
+ @io.puts "> Added the test case status"
98
+ @io.puts "> Submit Successful"
99
+ @io.close
100
+ return "Submitted to Test Rail"
101
+ end
102
+
103
+ def tr
104
+ @tr = ::TestRail::API.new( :user => @config.user,
105
+ :password => @config.password,
106
+ :namespace => @config.namespace)
107
+ end
108
+
109
+ # Add status to each testcase
110
+ def add_case_status(run_id)
111
+ @case_status.each { |case_id, status|
112
+ @tr.add_result_for_case(:run_id => run_id, :case_id => case_id, :status => status)
113
+ }
114
+ end
115
+
116
+ # Add section or cases within parent section
117
+ def create_suite(ir, project_id, suite, parent)
118
+ ir[:children].each do |child|
119
+ if @mappings.context.include?(child[:type])
120
+ section = parent.find_or_create_section(:project_id => project_id, :suite_id => suite.id, :name => child[:name])
121
+ create_suite(child, project_id, suite, section) if child[:children].count > 0
122
+
123
+ elsif @mappings.case.include?(child[:type])
124
+ parent = suite if parent.nil?
125
+ tcase = parent.find_or_create_test_case(:title => child[:name])
126
+ @case_status[:"#{tcase.id}"] = child[:status]
127
+
128
+ else
129
+ # To be added. Ex: steps
130
+ end # if
131
+ end # each
132
+ end # create_suite
133
+
134
+ def case_details(ir, section)
135
+ ir[:children].each do |child|
136
+ if @mappings.context.include?(child[:type])
137
+ section = section.find_section(:name => child[:name])
138
+ case_details(child, section)
139
+
140
+ elsif @mappings.case.include?(child[:type])
141
+ tcase = section.find_test_case(:title => child[:name]) if !section.nil?
142
+ @case_status[:"#{tcase.id}"] = child[:status] if !tcase.nil?
143
+
144
+ end
145
+ end
146
+ end
147
+
148
+ end # TestRail
149
+ end # Reporters
150
+ end # Res
@@ -0,0 +1,48 @@
1
+ require 'res/config'
2
+
3
+ module Res
4
+ module Reporters
5
+ class Testmine
6
+
7
+ attr_accessor :url, :config
8
+
9
+ def initialize(args)
10
+ @url = args[:url]
11
+ @config = Res::Config.new([:project, :component, :suite, :version, :url, :target],
12
+ :optional => [:hive_job_id],
13
+ :pre_env => 'TESTMINE_')
14
+ config.process(args)
15
+ end
16
+
17
+ def submit_results(ir, args)
18
+ # Set missing project information
19
+ ir.project = config.project
20
+ ir.suite = config.suite
21
+ ir.target = config.target
22
+ ir.hive_job_id = config.hive_job_id
23
+
24
+ # Load world information into json hash
25
+ ir.world = {
26
+ :project => @config.project,
27
+ :component => @config.component,
28
+ :version => @config.version,
29
+ }
30
+
31
+ # Submit to testmine
32
+ uri = URI.parse(config.url)
33
+ net = Net::HTTP.new(uri.host, uri.port)
34
+ request = Net::HTTP::Post.new("/api/v1/submit")
35
+ request.set_form_data({"data" => ir.to_json})
36
+ net.read_timeout = 60
37
+ net.open_timeout = 10
38
+
39
+ response = net.start do |http|
40
+ http.request(request)
41
+ end
42
+
43
+ response.read_body
44
+ end
45
+
46
+ end
47
+ end
48
+ end
metadata ADDED
@@ -0,0 +1,113 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: res
3
+ version: !ruby/object:Gem::Version
4
+ version: 1.0.0
5
+ platform: ruby
6
+ authors:
7
+ - BBC
8
+ - David Buckhurst
9
+ - Asim Khan
10
+ autorequire:
11
+ bindir: bin
12
+ cert_chain: []
13
+ date: 2015-10-09 00:00:00.000000000 Z
14
+ dependencies:
15
+ - !ruby/object:Gem::Dependency
16
+ name: json
17
+ requirement: !ruby/object:Gem::Requirement
18
+ requirements:
19
+ - - "~>"
20
+ - !ruby/object:Gem::Version
21
+ version: '1.8'
22
+ type: :runtime
23
+ prerelease: false
24
+ version_requirements: !ruby/object:Gem::Requirement
25
+ requirements:
26
+ - - "~>"
27
+ - !ruby/object:Gem::Version
28
+ version: '1.8'
29
+ - !ruby/object:Gem::Dependency
30
+ name: test_rail-api
31
+ requirement: !ruby/object:Gem::Requirement
32
+ requirements:
33
+ - - "~>"
34
+ - !ruby/object:Gem::Version
35
+ version: '0.4'
36
+ type: :runtime
37
+ prerelease: false
38
+ version_requirements: !ruby/object:Gem::Requirement
39
+ requirements:
40
+ - - "~>"
41
+ - !ruby/object:Gem::Version
42
+ version: '0.4'
43
+ - !ruby/object:Gem::Dependency
44
+ name: ox
45
+ requirement: !ruby/object:Gem::Requirement
46
+ requirements:
47
+ - - "~>"
48
+ - !ruby/object:Gem::Version
49
+ version: '2.2'
50
+ type: :runtime
51
+ prerelease: false
52
+ version_requirements: !ruby/object:Gem::Requirement
53
+ requirements:
54
+ - - "~>"
55
+ - !ruby/object:Gem::Version
56
+ version: '2.2'
57
+ - !ruby/object:Gem::Dependency
58
+ name: rspec
59
+ requirement: !ruby/object:Gem::Requirement
60
+ requirements:
61
+ - - "~>"
62
+ - !ruby/object:Gem::Version
63
+ version: '3.2'
64
+ type: :development
65
+ prerelease: false
66
+ version_requirements: !ruby/object:Gem::Requirement
67
+ requirements:
68
+ - - "~>"
69
+ - !ruby/object:Gem::Version
70
+ version: '3.2'
71
+ description: Formatters, parsers, and submitters for test results
72
+ email:
73
+ - david.buckhurst@bbc.co.uk
74
+ - asim.khan.ext@bbc.co.uk
75
+ executables: []
76
+ extensions: []
77
+ extra_rdoc_files: []
78
+ files:
79
+ - README.md
80
+ - lib/res.rb
81
+ - lib/res/config.rb
82
+ - lib/res/formatters/ruby_cucumber.rb
83
+ - lib/res/ir.rb
84
+ - lib/res/mappings.rb
85
+ - lib/res/parsers/junit.rb
86
+ - lib/res/reporters/hive.rb
87
+ - lib/res/reporters/test_rail.rb
88
+ - lib/res/reporters/testmine.rb
89
+ homepage: https://github.com/bbc/res
90
+ licenses:
91
+ - MIT
92
+ metadata: {}
93
+ post_install_message:
94
+ rdoc_options: []
95
+ require_paths:
96
+ - lib
97
+ required_ruby_version: !ruby/object:Gem::Requirement
98
+ requirements:
99
+ - - ">="
100
+ - !ruby/object:Gem::Version
101
+ version: '0'
102
+ required_rubygems_version: !ruby/object:Gem::Requirement
103
+ requirements:
104
+ - - ">="
105
+ - !ruby/object:Gem::Version
106
+ version: '0'
107
+ requirements: []
108
+ rubyforge_project:
109
+ rubygems_version: 2.4.8
110
+ signing_key:
111
+ specification_version: 4
112
+ summary: Test Result report libraries
113
+ test_files: []