rightscale_selfservice 0.0.3 → 0.0.4
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +8 -8
- data/README.md +151 -3
- data/lib/rightscale_selfservice/api/client.rb +17 -0
- data/lib/rightscale_selfservice/api/resource.rb +10 -0
- data/lib/rightscale_selfservice/cli/execution.rb +22 -0
- data/lib/rightscale_selfservice/cli/main.rb +22 -0
- data/lib/rightscale_selfservice/cli/operation.rb +26 -0
- data/lib/rightscale_selfservice/test/case.rb +155 -0
- data/lib/rightscale_selfservice/test/report.rb +44 -0
- data/lib/rightscale_selfservice/test/shell_report.rb +118 -0
- data/lib/rightscale_selfservice/test/suite.rb +44 -0
- data/lib/rightscale_selfservice/test/template.rb +161 -0
- metadata +7 -2
checksums.yaml
CHANGED
@@ -1,15 +1,15 @@
|
|
1
1
|
---
|
2
2
|
!binary "U0hBMQ==":
|
3
3
|
metadata.gz: !binary |-
|
4
|
-
|
4
|
+
MmMxMmMxYzA0NDNmNzk2NWVjODZmNjRkYjBlZTc4ODJmZDZmMmM3YQ==
|
5
5
|
data.tar.gz: !binary |-
|
6
|
-
|
6
|
+
MGUyNmQ0ZGZiZTAzNjllZmNjM2NiMzBlNjU2ZGE1ZjJlMjViMGRjZQ==
|
7
7
|
SHA512:
|
8
8
|
metadata.gz: !binary |-
|
9
|
-
|
10
|
-
|
11
|
-
|
9
|
+
ODg5NTRhOTE2MmNkYzMzZjExMWY0ZDg0YzZmYTg1ZTE4ODE5ZTVjYzQ3MDYx
|
10
|
+
YTc5OWJhNDljMDA5ZjYxN2EwMmMxNWY2ZjQyYmZhZTgyMThjZjA2ZGY2YTU3
|
11
|
+
M2M1ZjliODVjZDU3YTBkMjdlZTM5NTU1NmM5NzFlNGNmMTNlYzQ=
|
12
12
|
data.tar.gz: !binary |-
|
13
|
-
|
14
|
-
|
15
|
-
|
13
|
+
OTcxMjc0N2FlZDJjNzNiMWYyYzllYzliY2NlNmJkMGM4MGMyZDRkOWQyM2Vi
|
14
|
+
ODIzMGY0NWFlOTZhYjYzYzI3N2I0OWJkNzJjMzAzZWU3NmZmMjZjNTFmNTQ0
|
15
|
+
NzIwMjVjMGQyOWI4MjI3ZjMzZTY5ODM1NzQ3ZDQxMjZhNTM1ODM=
|
data/README.md
CHANGED
@@ -1,3 +1,36 @@
|
|
1
|
+
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
|
2
|
+
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
|
3
|
+
**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)*
|
4
|
+
|
5
|
+
- [rightscale_selfservice](#rightscale_selfservice)
|
6
|
+
- [Quick Start](#quick-start)
|
7
|
+
- [Authentication](#authentication)
|
8
|
+
- [CLI](#cli)
|
9
|
+
- [Template Includes](#template-includes)
|
10
|
+
- [Testing](#testing)
|
11
|
+
- [Suite](#suite)
|
12
|
+
- [Template](#template)
|
13
|
+
- [Case](#case)
|
14
|
+
- [Compile](#compile)
|
15
|
+
- [Execute](#execute)
|
16
|
+
- [Operations](#operations)
|
17
|
+
- [API Client](#api-client)
|
18
|
+
- [Supported Services, Resources and Actions](#supported-services-resources-and-actions)
|
19
|
+
- [How it works](#how-it-works)
|
20
|
+
- [Href Tokens](#href-tokens)
|
21
|
+
- [Automatic Multipart](#automatic-multipart)
|
22
|
+
- [Some examples...](#some-examples)
|
23
|
+
- [Authentication](#authentication-1)
|
24
|
+
- [Service Versions](#service-versions)
|
25
|
+
- [Executing Actions](#executing-actions)
|
26
|
+
- [List operations](#list-operations)
|
27
|
+
- [Show a template](#show-a-template)
|
28
|
+
- [Get RestClient::Request Instead of Executing Action](#get-restclientrequest-instead-of-executing-action)
|
29
|
+
- [Format Errors](#format-errors)
|
30
|
+
- [Multipart Detection](#multipart-detection)
|
31
|
+
|
32
|
+
<!-- END doctoc generated TOC please keep comment here to allow auto update -->
|
33
|
+
|
1
34
|
rightscale_selfservice
|
2
35
|
======================
|
3
36
|
|
@@ -28,13 +61,13 @@ properties as [right_api_client](https://github.com/rightscale/right_api_client)
|
|
28
61
|
|
29
62
|
The only additional required property is "selfservice_url".
|
30
63
|
|
31
|
-
The CLI can use a
|
64
|
+
The CLI can use a .yml file containing those properties or have them passed in
|
32
65
|
on the commandline. The API Client accepts an input hash with these same
|
33
66
|
properties.
|
34
67
|
|
35
|
-
An example
|
68
|
+
An example .yml file can be found [here](https://github.com/rightscale/right_api_client/blob/v1.5.24/config/login.yml.example)
|
36
69
|
|
37
|
-
A working
|
70
|
+
A working .yml file might look like;
|
38
71
|
```
|
39
72
|
:account_id: 12345
|
40
73
|
:email: user@domain.com
|
@@ -105,6 +138,121 @@ If that works, maybe start up a Cloud App from it
|
|
105
138
|
rightscale_selfservice template execute ~/Code/cat/somecat.cat.rb --auth-file=~/.right_api_client/login.yml
|
106
139
|
```
|
107
140
|
|
141
|
+
## Testing
|
142
|
+
|
143
|
+
Included as a framework for performing functional tests of your Cloud
|
144
|
+
Application Templates (CATs). Tests are performed as part of a [Suite](#suite)
|
145
|
+
which contains one or more [Template](#template), which have one or more test
|
146
|
+
[Case](#case).
|
147
|
+
|
148
|
+
### Suite
|
149
|
+
A suite is currently defined simply by the [glob](http://ruby-doc.org/core-1.9.3/Dir.html#method-c-glob)
|
150
|
+
you pass either to the [CLI](#cli) or the test suite class.
|
151
|
+
|
152
|
+
I might consider creating a config file or other options, but for now the suite
|
153
|
+
is basically just a container.
|
154
|
+
|
155
|
+
The suite will interrogate the files found by the input glob, and create
|
156
|
+
individual [Template](#template) classes.
|
157
|
+
|
158
|
+
### Template
|
159
|
+
A template is a 1-to-1 match with a CAT file found by the [Suite](#suite).
|
160
|
+
|
161
|
+
The template class will interrogate the contents of the CAT and create individual
|
162
|
+
[Case](#case) classes.
|
163
|
+
|
164
|
+
### Case
|
165
|
+
There are three types of cases, which are defined by comments in the CAT.
|
166
|
+
|
167
|
+
#### Compile
|
168
|
+
Merely checks if the CAT will compile successfully. When a template is tagged
|
169
|
+
to include a compile test case every other test will be ignored.
|
170
|
+
|
171
|
+
To create a test template which will only be tested for successful compile add
|
172
|
+
the following to the top of the template
|
173
|
+
```
|
174
|
+
#test:compile_only=true
|
175
|
+
```
|
176
|
+
|
177
|
+
#### Execute
|
178
|
+
This checks if the CAT will execute and reach a desired state.
|
179
|
+
|
180
|
+
If no tags are added to a template, an execute test case will be automatically
|
181
|
+
run, with the assumption that it's expected execution_state is "running".
|
182
|
+
|
183
|
+
To create a test template with an execute test case which will succeed if the
|
184
|
+
CAT launches successfully.
|
185
|
+
|
186
|
+
```
|
187
|
+
#test:execution_state=running
|
188
|
+
```
|
189
|
+
|
190
|
+
For the negative test case, where a CAT which launches and fails is actually a
|
191
|
+
successful test.
|
192
|
+
|
193
|
+
```
|
194
|
+
#test:execution_state=failed
|
195
|
+
```
|
196
|
+
|
197
|
+
There are also scenarios where a successful test should end in a specified state
|
198
|
+
but because of external issues (system bugs, reusable definitions, etc) the
|
199
|
+
opposite state is reached. In this case you can specify both the desired state
|
200
|
+
as the "execution_state" as well as an "execution_alternate_state".
|
201
|
+
|
202
|
+
If the CAT launches and reaches the alternate state the test case will be marked
|
203
|
+
as "FAILED (EXPECTED)", and if it reaches the desired state it will be marked
|
204
|
+
"FIXED"
|
205
|
+
|
206
|
+
```
|
207
|
+
#test:execution_state=running
|
208
|
+
#test:execution_alternate_state=failed
|
209
|
+
```
|
210
|
+
|
211
|
+
#### Operations
|
212
|
+
Once a template has been successfully launched, any operation test cases will
|
213
|
+
be run.
|
214
|
+
|
215
|
+
An example of a CAT with two operations which should be run as test cases;
|
216
|
+
```
|
217
|
+
#test:execution_state=running
|
218
|
+
#test:execution_alternate_state=failed
|
219
|
+
|
220
|
+
name "Foo"
|
221
|
+
rs_ca_ver 20131202
|
222
|
+
short_description "Foo"
|
223
|
+
|
224
|
+
#test_operation:execution_state=completed
|
225
|
+
#test_operation:execution_alternate_state=failed
|
226
|
+
#test_operation_param:key=val
|
227
|
+
#test_operation_param:key1=val1
|
228
|
+
#test_operation_param:key2=val2
|
229
|
+
operation "one" do
|
230
|
+
description "one"
|
231
|
+
definition "foo"
|
232
|
+
end
|
233
|
+
|
234
|
+
#test_operation:execution_state=completed
|
235
|
+
#test_operation:execution_alternate_state=failed
|
236
|
+
#test_operation_param:key=val
|
237
|
+
#test_operation_param:key1=val1
|
238
|
+
#test_operation_param:key2=val2
|
239
|
+
operation "two" do
|
240
|
+
description "two"
|
241
|
+
definition "foo"
|
242
|
+
end
|
243
|
+
|
244
|
+
define foo() do
|
245
|
+
|
246
|
+
end
|
247
|
+
```
|
248
|
+
|
249
|
+
This will run the operations named "one" and "two" with the specified
|
250
|
+
test_operation_param(s) as inputs to the operation. It will evaluate success
|
251
|
+
or failure based on the "execution_state" specified.
|
252
|
+
|
253
|
+
This has the same functionality as [Execute](#execute) in that you can specify
|
254
|
+
a desired and alternative state.
|
255
|
+
|
108
256
|
## API Client
|
109
257
|
|
110
258
|
The [CLI](#cli) bits consume the API client which is deployed with this gem. The
|
@@ -238,6 +238,23 @@ module RightScaleSelfService
|
|
238
238
|
url.gsub!(@selfservice_url,"")
|
239
239
|
end
|
240
240
|
|
241
|
+
# Accepts a full URL with protocol and hostname, or a relative href and
|
242
|
+
# returns the ID which is the last token in the path.
|
243
|
+
#
|
244
|
+
# I.E. https://hostname/api/foo/bar/baz/12345 == 12345
|
245
|
+
# and /api/foo/bar/baz/12345 == 12345
|
246
|
+
#
|
247
|
+
# @param url_href_or_id [String] The full url with protocol and hostname,
|
248
|
+
# a relative resource href, or the id.
|
249
|
+
#
|
250
|
+
# @return [String] ID of the resource referred to by the url or href, or
|
251
|
+
# nil if the id can not be determined.
|
252
|
+
def self.get_resource_id_from_href(url_href_or_id)
|
253
|
+
return url_href_or_id if url_href_or_id =~ /^[a-zA-Z0-9]*$/
|
254
|
+
matchdata = url_href_or_id.match(/[a-zA-Z0-9\/]*\/(?<id>[a-zA-Z0-9]*)$/)
|
255
|
+
matchdata['id'] if matchdata
|
256
|
+
end
|
257
|
+
|
241
258
|
# Accepts various possible responses and formats it into useful error text
|
242
259
|
#
|
243
260
|
# @param [RestClient::ExceptionWithResponse] error The response or error
|
@@ -64,8 +64,18 @@ module RightScaleSelfService
|
|
64
64
|
params[:payload] = args[0]
|
65
65
|
else
|
66
66
|
params[:payload] = URI.encode_www_form(args[0])
|
67
|
+
args[0].each do |k,v|
|
68
|
+
if v.is_a?(Array)
|
69
|
+
params[:payload].gsub!("#{k}=","#{k}[]=")
|
70
|
+
end
|
71
|
+
end
|
67
72
|
end
|
68
73
|
end
|
74
|
+
|
75
|
+
if method == :get && params.has_key?(:payload)
|
76
|
+
params[:url] += "?#{params[:payload]}"
|
77
|
+
params.delete(:payload)
|
78
|
+
end
|
69
79
|
end
|
70
80
|
|
71
81
|
request = @service.client.get_authorized_rest_client_request(params)
|
@@ -43,6 +43,28 @@ module RightScaleSelfService
|
|
43
43
|
logger.error(shell.set_color message, :red)
|
44
44
|
end
|
45
45
|
end
|
46
|
+
|
47
|
+
desc "show <id_or_href>", "Gets details about an execution (CloudApp) specified by <id_or_href>"
|
48
|
+
option :view, :type => :string, :default => 'default', :desc => "Which view to use, one of [default,expanded,source] default is 'default'."
|
49
|
+
option :property, :type => :array, :desc => "When supplied, only the specified properties will be displayed. By default the entire response is supplied. Ignored if view is 'source'"
|
50
|
+
def show(id_or_href)
|
51
|
+
client = get_api_client()
|
52
|
+
id = RightScaleSelfService::Api::Client.get_resource_id_from_href(id_or_href)
|
53
|
+
|
54
|
+
begin
|
55
|
+
response = client.manager.execution.show(:id => id, :view => @options["view"])
|
56
|
+
execution = JSON.parse(response.body)
|
57
|
+
if @options["property"] && @options["view"] != "source"
|
58
|
+
execution.delete_if{|k,v| !(@options["property"].include?(k))}
|
59
|
+
end
|
60
|
+
puts JSON.pretty_generate(execution)
|
61
|
+
rescue RestClient::ExceptionWithResponse => e
|
62
|
+
shell = Thor::Shell::Color.new
|
63
|
+
message = "Failed to show execution id #{id}\n\n#{RightScaleSelfService::Api::Client.format_error(e)}"
|
64
|
+
logger.error(shell.set_color message, :red)
|
65
|
+
end
|
66
|
+
end
|
67
|
+
|
46
68
|
end
|
47
69
|
end
|
48
70
|
end
|
@@ -33,6 +33,28 @@ module RightScaleSelfService
|
|
33
33
|
|
34
34
|
desc "operation", "Self Service Operation Commands"
|
35
35
|
subcommand "operation", Operation
|
36
|
+
|
37
|
+
desc "test <glob>", "Run a CAT Test suite consisting of files found in <glob>"
|
38
|
+
def test(glob)
|
39
|
+
client = get_api_client()
|
40
|
+
suite = RightScaleSelfService::Test::Suite.new(client, glob)
|
41
|
+
report = RightScaleSelfService::Test::ShellReport.new(suite)
|
42
|
+
begin
|
43
|
+
if suite.pump
|
44
|
+
report.progress
|
45
|
+
sleep(10)
|
46
|
+
else
|
47
|
+
break
|
48
|
+
end
|
49
|
+
end while true
|
50
|
+
report.progress
|
51
|
+
puts "\n\n"
|
52
|
+
error_text = report.errors
|
53
|
+
if error_text != ""
|
54
|
+
puts "\n\n"
|
55
|
+
end
|
56
|
+
report.failures
|
57
|
+
end
|
36
58
|
end
|
37
59
|
end
|
38
60
|
end
|
@@ -44,6 +44,32 @@ module RightScaleSelfService
|
|
44
44
|
logger.error(shell.set_color message, :red)
|
45
45
|
end
|
46
46
|
end
|
47
|
+
|
48
|
+
desc "list", "Lists all operations, optionally filtered by --filter and/or --property"
|
49
|
+
option :filter, :type => :array, :desc => "Filters to apply see (https://s3.amazonaws.com/rs_api_docs/selfservice/manager/index.html#/1.0/controller/V1::Controller::Operation/index)"
|
50
|
+
option :property, :type => :array, :desc => "When supplied, only the specified properties will be displayed. By default the entire response is supplied."
|
51
|
+
def list()
|
52
|
+
params = {}
|
53
|
+
if @options["filter"]
|
54
|
+
params[:filter] = @options["filter"]
|
55
|
+
end
|
56
|
+
|
57
|
+
begin
|
58
|
+
client = get_api_client()
|
59
|
+
response = client.manager.operation.index(params)
|
60
|
+
operations = JSON.parse(response.body)
|
61
|
+
if @options["property"]
|
62
|
+
operations.each do |op|
|
63
|
+
op.delete_if{|k,v| !(@options["property"].include?(k))}
|
64
|
+
end
|
65
|
+
end
|
66
|
+
puts JSON.pretty_generate(operations)
|
67
|
+
rescue RestClient::ExceptionWithResponse => e
|
68
|
+
shell = Thor::Shell::Color.new
|
69
|
+
message = "Failed to list operations\n\n#{RightScaleSelfService::Api::Client.format_error(e)}"
|
70
|
+
logger.error(shell.set_color message, :red)
|
71
|
+
end
|
72
|
+
end
|
47
73
|
end
|
48
74
|
end
|
49
75
|
end
|
@@ -0,0 +1,155 @@
|
|
1
|
+
# Copyright (c) 2014 Ryan Geyer
|
2
|
+
#
|
3
|
+
# Permission is hereby granted, free of charge, to any person obtaining a copy of
|
4
|
+
# this software and associated documentation files (the "Software"), to deal in
|
5
|
+
# the Software without restriction, including without limitation the rights to
|
6
|
+
# use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
|
7
|
+
# the Software, and to permit persons to whom the Software is furnished to do so,
|
8
|
+
# subject to the following conditions:
|
9
|
+
#
|
10
|
+
# The above copyright notice and this permission notice shall be included in all
|
11
|
+
# copies or substantial portions of the Software.
|
12
|
+
#
|
13
|
+
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
14
|
+
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
15
|
+
# FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
16
|
+
# COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
|
17
|
+
# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
|
18
|
+
# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
19
|
+
|
20
|
+
module RightScaleSelfService
|
21
|
+
module Test
|
22
|
+
class Case
|
23
|
+
attr_accessor :type
|
24
|
+
attr_accessor :options
|
25
|
+
attr_accessor :result
|
26
|
+
attr_accessor :errors
|
27
|
+
attr_accessor :finished
|
28
|
+
attr_accessor :failures
|
29
|
+
# initialized -> running -> completed | ?? -> finished
|
30
|
+
attr_accessor :state
|
31
|
+
attr_accessor :api_responses
|
32
|
+
|
33
|
+
def initialize(type, options = {})
|
34
|
+
self.type = type
|
35
|
+
self.options = options
|
36
|
+
self.result = ""
|
37
|
+
self.errors = []
|
38
|
+
self.failures = []
|
39
|
+
self.state = 'initialized'
|
40
|
+
self.api_responses = {}
|
41
|
+
end
|
42
|
+
|
43
|
+
# Performs the test for this case. Returns a boolean indicating if the
|
44
|
+
# test is "done" or not.
|
45
|
+
#
|
46
|
+
# @param suite [RightScaleSelfService::Test::Suite] The suite this test
|
47
|
+
# case belongs to
|
48
|
+
# @param template [RightScaleSelfService::Test::Template] The template
|
49
|
+
# this case belongs to
|
50
|
+
#
|
51
|
+
# @return [bool] True if this case needs to be pumped again, false if
|
52
|
+
# the case has been executed and does not need any further time slices
|
53
|
+
def pump(suite, template)
|
54
|
+
if result != ""
|
55
|
+
false
|
56
|
+
else
|
57
|
+
case self.type
|
58
|
+
when :compile_only
|
59
|
+
begin
|
60
|
+
suite.api_client.designer.template.compile(:source => template.template_string)
|
61
|
+
self.result = 'SUCCESS'
|
62
|
+
rescue RestClient::ExceptionWithResponse => e
|
63
|
+
if e.http_code == 422
|
64
|
+
self.result = 'FAILED'
|
65
|
+
self.failures << "Failed to compile template\n\n#{RightScaleSelfService::Api::Client.format_error(e)}"
|
66
|
+
else
|
67
|
+
self.result = 'ERROR'
|
68
|
+
self.errors << RightScaleSelfService::Api::Client.format_error(e)
|
69
|
+
end
|
70
|
+
end
|
71
|
+
false
|
72
|
+
when :execution
|
73
|
+
if self.options[:state] == template.state
|
74
|
+
self.result = 'SUCCESS'
|
75
|
+
self.result = 'FIXED' if self.options.has_key?(:alternate_state)
|
76
|
+
else
|
77
|
+
if self.options.has_key?(:alternate_state) && self.options[:alternate_state] == template.state
|
78
|
+
self.result = 'FAILED (EXPECTED)'
|
79
|
+
else
|
80
|
+
self.result = 'FAILED'
|
81
|
+
self.failures << "Expected execution end state to be (#{self.options[:state]}) but got execution end state (#{template.state})"
|
82
|
+
end
|
83
|
+
end
|
84
|
+
false
|
85
|
+
when :operation
|
86
|
+
if template.state == 'running'
|
87
|
+
case self.state
|
88
|
+
when 'initialized'
|
89
|
+
execution_id = template.execution_id
|
90
|
+
create_params = {
|
91
|
+
:execution_id => execution_id,
|
92
|
+
:name => self.options[:operation_name]
|
93
|
+
}
|
94
|
+
|
95
|
+
if self.options.has_key?(:params)
|
96
|
+
create_params[:options] = self.options[:params]
|
97
|
+
end
|
98
|
+
|
99
|
+
begin
|
100
|
+
self.api_responses[:operation_create] = suite.api_client.manager.operation.create(create_params)
|
101
|
+
self.state = 'running'
|
102
|
+
rescue RestClient::ExceptionWithResponse => e
|
103
|
+
self.errors << "Failed to create operation #{self.options[:operation_name]} for execution\n\n#{RightScaleSelfService::Api::Client.format_error(e)}"
|
104
|
+
self.result = 'ERROR'
|
105
|
+
return false
|
106
|
+
end
|
107
|
+
true
|
108
|
+
when 'running'
|
109
|
+
begin
|
110
|
+
operation_id = RightScaleSelfService::Api::Client.get_resource_id_from_href(self.api_responses[:operation_create].headers[:location])
|
111
|
+
if operation_id
|
112
|
+
self.api_responses[:operation_show] = suite.api_client.manager.operation.show(:id => operation_id)
|
113
|
+
json_str = self.api_responses[:operation_show].body
|
114
|
+
show = JSON.parse(json_str)
|
115
|
+
self.state = show['status']['summary']
|
116
|
+
end
|
117
|
+
rescue RestClient::ExceptionWithResponse => e
|
118
|
+
# TODO: Do I want to catch errors here, or let it fall through and
|
119
|
+
# let the next pump retry?
|
120
|
+
self.errors << "Failed to check operation status\n\n#{RightScaleSelfService::Api::Client.format_error(e)}"
|
121
|
+
end
|
122
|
+
when 'completed','failed'
|
123
|
+
if self.options[:state] == self.state
|
124
|
+
self.result = 'SUCCESS'
|
125
|
+
self.result = 'FIXED' if self.options.has_key?(:alternate_state)
|
126
|
+
else
|
127
|
+
if self.options.has_key?(:alternate_state) && self.options[:alternate_state] == self.state
|
128
|
+
self.result = 'FAILED (EXPECTED)'
|
129
|
+
else
|
130
|
+
self.result = 'FAILED'
|
131
|
+
self.failures << "Expected operation end state to be (#{self.options[:state]}) but got execution end state (#{self.state})"
|
132
|
+
end
|
133
|
+
end
|
134
|
+
false
|
135
|
+
else
|
136
|
+
end
|
137
|
+
elsif template.state == 'failed'
|
138
|
+
self.result = 'FAILED'
|
139
|
+
self.failures << "Execution failed to start, could not start a new operation."
|
140
|
+
false
|
141
|
+
else
|
142
|
+
self.result = 'ERROR'
|
143
|
+
self.errors << "Unexpected execution state #{template.state} while processing operation test case."
|
144
|
+
false
|
145
|
+
end
|
146
|
+
else
|
147
|
+
self.result = 'ERROR'
|
148
|
+
self.errors << "Unknown test case type (:#{self.type})"
|
149
|
+
false
|
150
|
+
end
|
151
|
+
end
|
152
|
+
end
|
153
|
+
end
|
154
|
+
end
|
155
|
+
end
|
@@ -0,0 +1,44 @@
|
|
1
|
+
# Copyright (c) 2014 Ryan Geyer
|
2
|
+
#
|
3
|
+
# Permission is hereby granted, free of charge, to any person obtaining a copy of
|
4
|
+
# this software and associated documentation files (the "Software"), to deal in
|
5
|
+
# the Software without restriction, including without limitation the rights to
|
6
|
+
# use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
|
7
|
+
# the Software, and to permit persons to whom the Software is furnished to do so,
|
8
|
+
# subject to the following conditions:
|
9
|
+
#
|
10
|
+
# The above copyright notice and this permission notice shall be included in all
|
11
|
+
# copies or substantial portions of the Software.
|
12
|
+
#
|
13
|
+
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
14
|
+
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
15
|
+
# FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
16
|
+
# COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
|
17
|
+
# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
|
18
|
+
# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
19
|
+
|
20
|
+
module RightScaleSelfService
|
21
|
+
module Test
|
22
|
+
class Report
|
23
|
+
attr_accessor :suite
|
24
|
+
attr_accessor :options
|
25
|
+
|
26
|
+
def initialize(suite, options={})
|
27
|
+
self.suite = suite
|
28
|
+
self.options = options
|
29
|
+
end
|
30
|
+
|
31
|
+
def progress
|
32
|
+
raise NotImplementedError.new()
|
33
|
+
end
|
34
|
+
|
35
|
+
def errors
|
36
|
+
raise NotImplementedError.new()
|
37
|
+
end
|
38
|
+
|
39
|
+
def failures
|
40
|
+
raise NotImplementedError.new()
|
41
|
+
end
|
42
|
+
end
|
43
|
+
end
|
44
|
+
end
|
@@ -0,0 +1,118 @@
|
|
1
|
+
# Copyright (c) 2014 Ryan Geyer
|
2
|
+
#
|
3
|
+
# Permission is hereby granted, free of charge, to any person obtaining a copy of
|
4
|
+
# this software and associated documentation files (the "Software"), to deal in
|
5
|
+
# the Software without restriction, including without limitation the rights to
|
6
|
+
# use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
|
7
|
+
# the Software, and to permit persons to whom the Software is furnished to do so,
|
8
|
+
# subject to the following conditions:
|
9
|
+
#
|
10
|
+
# The above copyright notice and this permission notice shall be included in all
|
11
|
+
# copies or substantial portions of the Software.
|
12
|
+
#
|
13
|
+
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
14
|
+
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
15
|
+
# FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
16
|
+
# COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
|
17
|
+
# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
|
18
|
+
# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
19
|
+
|
20
|
+
# Require report.rb first, to satisfy Travis CI
|
21
|
+
require File.expand_path(File.join(File.dirname(__FILE__), 'report'))
|
22
|
+
|
23
|
+
module RightScaleSelfService
|
24
|
+
module Test
|
25
|
+
class ShellReport < Report
|
26
|
+
|
27
|
+
def initialize(suite, options={})
|
28
|
+
super(suite,options)
|
29
|
+
@reported_templates = []
|
30
|
+
@keyword_substitutions = {
|
31
|
+
"SUCCESS" => "\e[42m\e[30mSUCCESS\e[0m",
|
32
|
+
"FAILED" => "\e[41m\e[30mFAILED\e[0m",
|
33
|
+
"FAILED (EXPECTED)" => "\e[43m\e[30mFAILED (EXPECTED)\e[0m",
|
34
|
+
"ERROR" => "\e[41m\e[30mERROR\e[0m",
|
35
|
+
"FIXED" => "\e[44m\e[30mFIXED\e[0m"
|
36
|
+
}
|
37
|
+
end
|
38
|
+
|
39
|
+
def progress
|
40
|
+
self.suite.templates.each do |template|
|
41
|
+
if template.state == "finished" && !@reported_templates.include?(template)
|
42
|
+
puts "#{template.name}: #{template.state}"
|
43
|
+
template.cases.each do |testcase|
|
44
|
+
puts " #{get_case_type_and_name(testcase)}: #{@keyword_substitutions[testcase.result]}"
|
45
|
+
end
|
46
|
+
@reported_templates << template
|
47
|
+
end
|
48
|
+
end
|
49
|
+
end
|
50
|
+
|
51
|
+
def errors
|
52
|
+
finished_templates = self.suite.templates.select{|t| t.state == "finished"}
|
53
|
+
if finished_templates.size == self.suite.templates.size
|
54
|
+
to_puts = ""
|
55
|
+
self.suite.templates.each do |template|
|
56
|
+
cases_with_errors = template.cases.select{|c| c.errors.size > 0}
|
57
|
+
if template.errors.size > 0 || cases_with_errors.size > 0
|
58
|
+
to_puts += "#{template.name}:\n"
|
59
|
+
end
|
60
|
+
template.errors.each do |error|
|
61
|
+
to_puts += " #{error}\n"
|
62
|
+
end
|
63
|
+
cases_with_errors.each do |case_with_errors|
|
64
|
+
to_puts += " #{get_case_type_and_name(case_with_errors)}:\n"
|
65
|
+
case_with_errors.errors.each do |error|
|
66
|
+
to_puts += " #{error}\n"
|
67
|
+
end
|
68
|
+
end
|
69
|
+
end
|
70
|
+
if to_puts.size > 0
|
71
|
+
puts "\e[41m\e[30mERRORS:\e[0m"
|
72
|
+
puts "\e[31m#{to_puts}\e[0m"
|
73
|
+
end
|
74
|
+
end
|
75
|
+
end
|
76
|
+
|
77
|
+
def failures
|
78
|
+
finished_templates = self.suite.templates.select{|t| t.state == "finished"}
|
79
|
+
to_puts = ""
|
80
|
+
if finished_templates.size == self.suite.templates.size
|
81
|
+
to_puts = ""
|
82
|
+
self.suite.templates.each do |template|
|
83
|
+
cases_with_failures = template.cases.select{|c| c.failures.size > 0}
|
84
|
+
if cases_with_failures.size > 0
|
85
|
+
to_puts += "#{template.name}:\n"
|
86
|
+
end
|
87
|
+
cases_with_failures.each do |case_with_fail|
|
88
|
+
to_puts += " #{get_case_type_and_name(case_with_fail)}:\n"
|
89
|
+
# TODO: Add details for an operation case type to differentiate
|
90
|
+
# them. This is probably an argument for having some of the
|
91
|
+
# progress, error, and failure reporting live in the case
|
92
|
+
# class, but we don't want the case class to have to know details
|
93
|
+
# about how to output. Hrrmnn.
|
94
|
+
case_with_fail.failures.each do |failure|
|
95
|
+
to_puts += " #{failure}\n"
|
96
|
+
end
|
97
|
+
end
|
98
|
+
end
|
99
|
+
if to_puts.size > 0
|
100
|
+
to_puts = "Failures:\n#{to_puts}"
|
101
|
+
puts to_puts
|
102
|
+
end
|
103
|
+
end
|
104
|
+
end
|
105
|
+
|
106
|
+
private
|
107
|
+
|
108
|
+
def get_case_type_and_name(test_case)
|
109
|
+
retval = test_case.type.to_s
|
110
|
+
if test_case.type == :operation
|
111
|
+
retval += " (#{test_case.options[:operation_name]})"
|
112
|
+
end
|
113
|
+
retval
|
114
|
+
end
|
115
|
+
|
116
|
+
end
|
117
|
+
end
|
118
|
+
end
|
@@ -0,0 +1,44 @@
|
|
1
|
+
# Copyright (c) 2014 Ryan Geyer
|
2
|
+
#
|
3
|
+
# Permission is hereby granted, free of charge, to any person obtaining a copy of
|
4
|
+
# this software and associated documentation files (the "Software"), to deal in
|
5
|
+
# the Software without restriction, including without limitation the rights to
|
6
|
+
# use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
|
7
|
+
# the Software, and to permit persons to whom the Software is furnished to do so,
|
8
|
+
# subject to the following conditions:
|
9
|
+
#
|
10
|
+
# The above copyright notice and this permission notice shall be included in all
|
11
|
+
# copies or substantial portions of the Software.
|
12
|
+
#
|
13
|
+
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
14
|
+
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
15
|
+
# FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
16
|
+
# COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
|
17
|
+
# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
|
18
|
+
# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
19
|
+
|
20
|
+
module RightScaleSelfService
|
21
|
+
module Test
|
22
|
+
class Suite
|
23
|
+
attr_accessor :api_client
|
24
|
+
attr_accessor :templates
|
25
|
+
|
26
|
+
def initialize(api_client, glob)
|
27
|
+
self.templates = []
|
28
|
+
self.api_client = api_client
|
29
|
+
template_files = Dir.glob(glob)
|
30
|
+
template_files.each do |template|
|
31
|
+
templates << RightScaleSelfService::Test::Template.new(template)
|
32
|
+
end
|
33
|
+
end
|
34
|
+
|
35
|
+
def pump
|
36
|
+
finished_templates = templates.select do |t|
|
37
|
+
t.pump(self)
|
38
|
+
t.state == 'finished'
|
39
|
+
end
|
40
|
+
templates.length != finished_templates.length
|
41
|
+
end
|
42
|
+
end
|
43
|
+
end
|
44
|
+
end
|
@@ -0,0 +1,161 @@
|
|
1
|
+
# Copyright (c) 2014 Ryan Geyer
|
2
|
+
#
|
3
|
+
# Permission is hereby granted, free of charge, to any person obtaining a copy of
|
4
|
+
# this software and associated documentation files (the "Software"), to deal in
|
5
|
+
# the Software without restriction, including without limitation the rights to
|
6
|
+
# use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
|
7
|
+
# the Software, and to permit persons to whom the Software is furnished to do so,
|
8
|
+
# subject to the following conditions:
|
9
|
+
#
|
10
|
+
# The above copyright notice and this permission notice shall be included in all
|
11
|
+
# copies or substantial portions of the Software.
|
12
|
+
#
|
13
|
+
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
14
|
+
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
15
|
+
# FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
16
|
+
# COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
|
17
|
+
# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
|
18
|
+
# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
19
|
+
|
20
|
+
module RightScaleSelfService
|
21
|
+
module Test
|
22
|
+
class Template
|
23
|
+
attr_accessor :cases
|
24
|
+
attr_accessor :template_string
|
25
|
+
# intialized -> launching -> running | failed -> terminating -> finished
|
26
|
+
attr_accessor :state
|
27
|
+
|
28
|
+
attr_accessor :errors
|
29
|
+
attr_accessor :api_responses
|
30
|
+
|
31
|
+
attr_reader :name
|
32
|
+
|
33
|
+
def initialize(filepath)
|
34
|
+
@name = File.basename(filepath)
|
35
|
+
self.api_responses = {}
|
36
|
+
self.state = "initialized"
|
37
|
+
self.errors = []
|
38
|
+
self.cases = []
|
39
|
+
self.template_string = template_str = RightScaleSelfService::Utilities::Template.preprocess(filepath)
|
40
|
+
test_config = {}
|
41
|
+
if template_str.include?('#test:compile_only=true')
|
42
|
+
self.cases = [RightScaleSelfService::Test::Case.new(:compile_only)]
|
43
|
+
self.state = "running"
|
44
|
+
else
|
45
|
+
execution_state_matches = template_str.match(/^#test:execution_state=(?<state>[0-9a-zA-Z ]*)/)
|
46
|
+
execution_state = 'running'
|
47
|
+
if execution_state_matches && execution_state_matches.names.include?('state')
|
48
|
+
execution_state = execution_state_matches['state']
|
49
|
+
end
|
50
|
+
options = {:state => execution_state}
|
51
|
+
alt_state_matches = template_str.match(/^#test:execution_alternate_state=(?<state>[0-9a-zA-Z ]*)/)
|
52
|
+
if alt_state_matches && alt_state_matches.size > 0
|
53
|
+
options[:alternate_state] = alt_state_matches['state']
|
54
|
+
end
|
55
|
+
self.cases << RightScaleSelfService::Test::Case.new(:execution, options)
|
56
|
+
|
57
|
+
template_str.scan(/(#test_operation:.*?)\noperation ["'](.*?)["'] do/m).each do |operation|
|
58
|
+
tags = operation[0]
|
59
|
+
operation_name = operation[1]
|
60
|
+
|
61
|
+
options = {:operation_name => operation_name}
|
62
|
+
|
63
|
+
execution_state_matches = tags.match(/^#test_operation:execution_state=(?<state>[0-9a-zA-Z ]*)/)
|
64
|
+
if execution_state_matches && execution_state_matches.names.include?('state')
|
65
|
+
options[:state] = execution_state_matches['state']
|
66
|
+
end
|
67
|
+
|
68
|
+
alt_state_matches = tags.match(/^#test_operation:execution_alternate_state=(?<state>[0-9a-zA-Z ]*)/)
|
69
|
+
if alt_state_matches && alt_state_matches.names.include?('state')
|
70
|
+
options[:alternate_state] = alt_state_matches['state']
|
71
|
+
end
|
72
|
+
|
73
|
+
tags.scan(/#test_operation_param:(?<key>.*?)=(?<val>.*?)$/).each do |param_pair|
|
74
|
+
options[:params] = {} unless options.has_key?(:params)
|
75
|
+
options[:params][param_pair[0]] = param_pair[1]
|
76
|
+
end
|
77
|
+
|
78
|
+
self.cases << RightScaleSelfService::Test::Case.new(:operation, options)
|
79
|
+
end
|
80
|
+
end
|
81
|
+
end
|
82
|
+
|
83
|
+
def pump(suite)
|
84
|
+
case self.state
|
85
|
+
when 'initialized'
|
86
|
+
begin
|
87
|
+
self.api_responses[:execution_create] = suite.api_client.manager.execution.create(:source => self.template_string)
|
88
|
+
self.state = 'launching'
|
89
|
+
rescue RestClient::ExceptionWithResponse => e
|
90
|
+
self.errors << "Failed to create execution from template\n\n#{RightScaleSelfService::Api::Client.format_error(e)}"
|
91
|
+
self.state = 'failed'
|
92
|
+
end
|
93
|
+
when 'launching'
|
94
|
+
get_execution_status_and_set_as_state(suite)
|
95
|
+
when 'running','failed'
|
96
|
+
# TODO: Any error handling here?
|
97
|
+
unfinished_cases = cases.select {|c| c.pump(suite, self)}
|
98
|
+
if unfinished_cases.size == 0
|
99
|
+
if self.api_responses.has_key?(:execution_create)
|
100
|
+
exec_id = execution_id()
|
101
|
+
self.api_responses[:terminate_operation_create] =
|
102
|
+
suite.api_client.manager.operation.create(
|
103
|
+
:name => 'terminate', :execution_id => exec_id
|
104
|
+
)
|
105
|
+
self.state = 'terminating'
|
106
|
+
else
|
107
|
+
self.state = 'terminated'
|
108
|
+
end
|
109
|
+
end
|
110
|
+
when 'terminating'
|
111
|
+
get_execution_status_and_set_as_state(suite)
|
112
|
+
when 'terminated'
|
113
|
+
begin
|
114
|
+
exec_id = execution_id()
|
115
|
+
if exec_id
|
116
|
+
suite.api_client.manager.execution.delete(:id => exec_id)
|
117
|
+
end
|
118
|
+
rescue RestClient::ExceptionWithResponse => e
|
119
|
+
self.errors << "Failed to delete execution #{self.api_response[:execution_create][:headers][:location]}\n\n#{RightScaleSelfService::Api::Client.format_error(e)}"
|
120
|
+
end
|
121
|
+
self.state = 'finished'
|
122
|
+
when 'finished'
|
123
|
+
# Do nothing
|
124
|
+
else
|
125
|
+
self.errors << "unknown template state #{self.state}"
|
126
|
+
self.state = 'finished'
|
127
|
+
end
|
128
|
+
end
|
129
|
+
|
130
|
+
def execution_id
|
131
|
+
unless self.api_responses.has_key?(:execution_create)
|
132
|
+
nil
|
133
|
+
else
|
134
|
+
execution_id = RightScaleSelfService::Api::Client
|
135
|
+
.get_resource_id_from_href(
|
136
|
+
self.api_responses[:execution_create].headers[:location]
|
137
|
+
)
|
138
|
+
end
|
139
|
+
end
|
140
|
+
|
141
|
+
private
|
142
|
+
|
143
|
+
def get_execution_status_and_set_as_state(suite)
|
144
|
+
begin
|
145
|
+
exec_id = execution_id()
|
146
|
+
if exec_id
|
147
|
+
self.api_responses[:execution_show] = suite.api_client.manager.execution.show(:id => exec_id)
|
148
|
+
json_str = self.api_responses[:execution_show].body
|
149
|
+
show = JSON.parse(json_str)
|
150
|
+
self.state = show['status']
|
151
|
+
end
|
152
|
+
rescue RestClient::ExceptionWithResponse => e
|
153
|
+
# TODO: Do I want to catch errors here, or let it fall through and
|
154
|
+
# let the next pump retry?
|
155
|
+
self.errors << "Failed to check execution status\n\n#{RightScaleSelfService::Api::Client.format_error(e)}"
|
156
|
+
end
|
157
|
+
end
|
158
|
+
|
159
|
+
end
|
160
|
+
end
|
161
|
+
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: rightscale_selfservice
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.0.
|
4
|
+
version: 0.0.4
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Ryan J. Geyer
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2015-01-23 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: thor
|
@@ -73,6 +73,11 @@ files:
|
|
73
73
|
- lib/rightscale_selfservice/cli/main.rb
|
74
74
|
- lib/rightscale_selfservice/cli/operation.rb
|
75
75
|
- lib/rightscale_selfservice/cli/template.rb
|
76
|
+
- lib/rightscale_selfservice/test/case.rb
|
77
|
+
- lib/rightscale_selfservice/test/report.rb
|
78
|
+
- lib/rightscale_selfservice/test/shell_report.rb
|
79
|
+
- lib/rightscale_selfservice/test/suite.rb
|
80
|
+
- lib/rightscale_selfservice/test/template.rb
|
76
81
|
- lib/rightscale_selfservice/utilities/template.rb
|
77
82
|
homepage: https://github.com/rgeyer/rightscale_selfservice
|
78
83
|
licenses:
|