clouds 0.1.3 → 0.1.6
Sign up to get free protection for your applications and to get access to all the features.
- data/README.md +23 -11
- data/bin/clouds +12 -4
- data/lib/clouds.rb +53 -17
- data/lib/clouds/version.rb +1 -1
- metadata +2 -2
data/README.md
CHANGED
@@ -1,6 +1,11 @@
|
|
1
1
|
# clouds
|
2
2
|
|
3
|
-
This is a simple tool that aims to ease the handling of [AWS CloudFormation](https://aws.amazon.com/cloudformation/) stacks, following the [infrastructure as code](http://sdarchitect.wordpress.com/2012/12/13/infrastructure-as-code/) philosophy in ways that are not possible with the AWS console. Even though you can achieve pretty much the same using the AWS command line tools, `clouds` aims to be much easier to use, cleaner and more specialized in handling CloudFormation stacks as code.
|
3
|
+
This is a simple tool that aims to ease the handling of [AWS CloudFormation](https://aws.amazon.com/cloudformation/) stacks, following the [infrastructure as code](http://sdarchitect.wordpress.com/2012/12/13/infrastructure-as-code/) philosophy in ways that are not possible with the AWS console. Even though you can achieve pretty much the same things using the AWS command line tools, `clouds` aims to be much easier to use, cleaner and more specialized in handling CloudFormation stacks as code.
|
4
|
+
|
5
|
+
It was developed at [HERE](http://here.com) and we use it pretty much on a daily basis for updating our existing stacks and creating new ones.
|
6
|
+
|
7
|
+
For some details about the story behind it, you can see this [presentation](http://slidesha.re/U7SRPq) on Slideshare.
|
8
|
+
|
4
9
|
|
5
10
|
## Features
|
6
11
|
- Upload templates and parameters from the current directory into AWS with a single intuitive command. They need to be located in stacks/stack_name/{template.json,parameters.yaml}
|
@@ -8,21 +13,28 @@ This is a simple tool that aims to ease the handling of [AWS CloudFormation](htt
|
|
8
13
|
- Dump one or all of the stacks from your currently selected AWS account into the current directory (preferably stored in a version control system) under the 'stacks' directory, including both the template JSON code and the parameters that were used for launching it, which will then be saved in a human-friendly YAML file.
|
9
14
|
- Perform updates on the AWS stacks once you modified the dumped data (template or parameters)
|
10
15
|
- Validate templates for JSON correctness when uploading or updating a stack. The error messages should help you debug syntax errors.
|
11
|
-
- Clone existing stacks to create new ones. It defaults to local clone, which then needs another update call to get in effect. This is an easy way to migrate infrastructure code and
|
16
|
+
- Clone existing stacks to create new ones. It defaults to a local clone, which then needs another update call to get in effect. This is an easy way to migrate infrastructure 'code' around and makes it trivial to promote changes between environments, and can also be used easily with projects running in different AWS accounts. This is especially convenient together with an AWS profile switcher shell alias like implemented on the oh-my-zsh aws plugin.
|
12
17
|
- Perform cost calculations when creating a new stack.
|
13
18
|
- Delete existing stacks with a simple command
|
14
|
-
- For authentication it can use AWS account credentials defined using either the AWS_ACCESS_KEY_ID&AWS_SECRET_ACCESS_KEY combination, or the AWS_DEFAULT_PROFILE environment
|
19
|
+
- For authentication it can use AWS account credentials defined using either the AWS_ACCESS_KEY_ID&AWS_SECRET_ACCESS_KEY combination, or the AWS_DEFAULT_PROFILE environment variable
|
15
20
|
- Able to use temporary credentials, if the AWS_SECURITY_TOKEN variable is defined in the environment.
|
16
21
|
|
17
22
|
## Installation
|
18
23
|
|
24
|
+
###On Ruby 1.9 or newer
|
25
|
+
|
19
26
|
gem install clouds
|
20
27
|
|
21
|
-
|
28
|
+
###On Ruby 1.8.x
|
29
|
+
|
30
|
+
You might need to install `rubygems` separtely and you should be aware of the incompatibility between the more recent versions of `nokogiri`(a dependency of the `aws-sdk`, on which `clouds` indirectly depends on) and Ruby 1.8. This is pretty well-explained [here](http://ruby.awsblog.com/post/Tx2T9MFQJK7U74N/AWS-SDK-for-Ruby-and-Nokogiri).
|
22
31
|
|
23
|
-
|
32
|
+
You will need to manually install the latest compatible version of nokogiri and its build-time dependencies, as documented below:
|
24
33
|
|
25
|
-
|
34
|
+
apt-get install libxml2-dev libxslt-dev # Debian and clones
|
35
|
+
yum install libxml2-devel libxslt-devel # RedHat and clones
|
36
|
+
gem install nokogiri --version="<1.6"
|
37
|
+
gem install clouds
|
26
38
|
|
27
39
|
## Running
|
28
40
|
Execute this in your shell:
|
@@ -53,7 +65,7 @@ Clone a stack (locally only, so you can perform some changes)
|
|
53
65
|
|
54
66
|
clouds clone stack new_stack
|
55
67
|
|
56
|
-
Upload the cloned stack to AWS CloudFormation
|
68
|
+
Upload the cloned stack to AWS CloudFormation. This can also be used against templates you downloaded from third other sources, not just for dumps.
|
57
69
|
|
58
70
|
clouds update new_stack -c
|
59
71
|
|
@@ -65,7 +77,7 @@ Delete the new stack (needs --force)
|
|
65
77
|
## Development
|
66
78
|
|
67
79
|
### Build requirements
|
68
|
-
- Ruby, rubygems and rake (some might already be there on Ruby > 1.9)
|
80
|
+
- Ruby, rubygems and rake (some of them might already be there on Ruby > 1.9)
|
69
81
|
- Development headers for libxml2 and libxslt (-devel packages)
|
70
82
|
- Bundler
|
71
83
|
|
@@ -83,15 +95,15 @@ Only in case you didn't do it before
|
|
83
95
|
### Updating the installed gem
|
84
96
|
|
85
97
|
rake repackage
|
86
|
-
sudo gem install pkg/clouds
|
98
|
+
sudo gem install pkg/clouds-*.gem
|
87
99
|
|
88
100
|
### Build instructions
|
89
101
|
|
90
|
-
|
102
|
+
Once you have the stated requirements satisfied, you can just package it as gem
|
91
103
|
|
92
104
|
rake package
|
93
105
|
|
94
106
|
### Installing your own gem
|
95
107
|
|
96
|
-
sudo gem install pkg/clouds
|
108
|
+
sudo gem install pkg/clouds-*.gem
|
97
109
|
|
data/bin/clouds
CHANGED
@@ -108,14 +108,22 @@ command :update do |c|
|
|
108
108
|
c.switch [:a,:all], :negatable => false
|
109
109
|
c.desc 'Create AWS stacks if the required sources exist locally'
|
110
110
|
c.switch [:c,:create_missing], :negatable => false
|
111
|
+
c.desc 'Wait until finished (synchronous mode)'
|
112
|
+
c.switch [:w, :wait], :negatable => false
|
113
|
+
c.desc 'Print stack outputs in yaml format'
|
114
|
+
c.switch [:o, :outputs], :negatable => false
|
111
115
|
|
112
116
|
c.action do |global_options,options,args|
|
113
117
|
if args.empty? && options[:all]
|
114
|
-
puts '
|
115
|
-
update_all_stacks(options[:create_missing]
|
118
|
+
puts '# updating all stacks'
|
119
|
+
raise 'Failed to update the stack' unless update_all_stacks(options[:create_missing],
|
120
|
+
options[:wait],
|
121
|
+
options[:outputs])
|
116
122
|
elsif args.length >= 1
|
117
|
-
puts "updating the following stacks: #{args.inspect}"
|
118
|
-
update_stacks(args,options[:create_missing]
|
123
|
+
puts "# updating the following stacks: #{args.inspect}"
|
124
|
+
raise 'Failed to update the stack' unless update_stacks(args,options[:create_missing],
|
125
|
+
options[:wait],
|
126
|
+
options[:outputs])
|
119
127
|
else # no arguments and --all flag
|
120
128
|
raise 'This command needs a list of stacks, or the --all flag in order to update all the stacks.'
|
121
129
|
end
|
data/lib/clouds.rb
CHANGED
@@ -140,20 +140,29 @@ def dump_all_stacks(force=false)
|
|
140
140
|
end
|
141
141
|
end
|
142
142
|
|
143
|
-
def update_stack(stack_name, create_if_missing=false)
|
144
|
-
configure
|
145
|
-
stack=nil
|
143
|
+
def update_stack(stack_name, create_if_missing=false, synchronous=false, outputs=false)
|
144
|
+
configure
|
145
|
+
stack = nil
|
146
146
|
|
147
147
|
template_content = read_file(get_template_path(stack_name))
|
148
148
|
parameters_content = read_file(get_parameters_path(stack_name))
|
149
149
|
|
150
|
+
parameters_hash = {}
|
151
|
+
|
150
152
|
begin
|
151
|
-
|
153
|
+
yaml_hash = YAML.load(parameters_content)
|
152
154
|
rescue => e
|
153
155
|
puts e
|
154
156
|
raise e
|
155
157
|
end
|
156
158
|
|
159
|
+
yaml_hash.each do |k, v|
|
160
|
+
v = v.join(",") if v.is_a?(Array)
|
161
|
+
parameters_hash[k] = v
|
162
|
+
end
|
163
|
+
|
164
|
+
p parameters_hash
|
165
|
+
|
157
166
|
raise 'Empty stack template' if template_content.nil? || template_content.empty?
|
158
167
|
|
159
168
|
template_validation = @cfn.validate_template(template_content)
|
@@ -161,43 +170,71 @@ def update_stack(stack_name, create_if_missing=false)
|
|
161
170
|
|
162
171
|
begin
|
163
172
|
if @cfn.stacks[stack_name].exists?
|
164
|
-
puts "Updating stack #{stack_name}"
|
173
|
+
puts "# Updating stack #{stack_name}"
|
165
174
|
stack = @cfn.stacks[stack_name]
|
166
175
|
stack_capabilities = stack.capabilities
|
167
176
|
stack.update(:template => template_content,
|
168
177
|
:parameters => parameters_hash,
|
169
178
|
:capabilities => stack_capabilities)
|
179
|
+
status = wait_until_status(stack, %w( CREATE_IN_PROGRESS UPDATE_IN_PROGRESS )) if synchronous
|
170
180
|
elsif create_if_missing
|
171
|
-
puts "Creating stack #{stack_name}"
|
181
|
+
puts "# Creating stack #{stack_name}"
|
172
182
|
stack = @cfn.stacks.create(stack_name,
|
173
183
|
template_content,
|
174
184
|
{ :parameters => parameters_hash,
|
175
185
|
:capabilities => ['CAPABILITY_IAM']})
|
186
|
+
status = wait_until_status(stack, %w( CREATE_IN_PROGRESS UPDATE_IN_PROGRESS )) if synchronous
|
176
187
|
else
|
177
188
|
puts "Skipping stack #{stack_name} since it's not defined in this AWS account, if the stack exists locally you might use the -c flag"
|
178
189
|
end
|
179
|
-
|
180
|
-
estimated_cost = stack.estimate_template_cost
|
181
|
-
puts "Estimated costs for the stack #{stack_name} is #{estimated_cost}"
|
190
|
+
if ! stack.nil? && %w( UPDATE_COMPLETE CREATE_COMPLETE ).include?(status)
|
191
|
+
estimated_cost = stack.estimate_template_cost
|
192
|
+
puts "# Estimated costs for the stack #{stack_name} is #{estimated_cost}"
|
193
|
+
|
194
|
+
print_stack_outputs(stack) if outputs
|
182
195
|
end
|
183
196
|
rescue => e
|
184
197
|
puts e
|
185
198
|
end
|
199
|
+
|
200
|
+
%w( UPDATE_COMPLETE CREATE_COMPLETE ).include? status
|
186
201
|
end
|
187
202
|
|
188
|
-
def
|
189
|
-
|
190
|
-
|
203
|
+
def wait_until_status(stack, status_arr)
|
204
|
+
while true
|
205
|
+
begin
|
206
|
+
status = stack.status
|
207
|
+
printf("# %s : %s\n", Time.now.strftime('%Y-%m-%d %H:%M:%S'), status)
|
208
|
+
return status unless status_arr.include? status
|
209
|
+
sleep 5
|
210
|
+
rescue => e
|
211
|
+
raise "Cannot retrieve stack status: #{e}"
|
212
|
+
end
|
191
213
|
end
|
192
214
|
end
|
193
215
|
|
194
|
-
def
|
195
|
-
|
196
|
-
|
197
|
-
|
216
|
+
def print_stack_outputs(stack)
|
217
|
+
puts '# ---'
|
218
|
+
puts "# Outputs from #{stack.name}"
|
219
|
+
stack.outputs.each do |o|
|
220
|
+
puts "#{o.key}: #{o.value}"
|
198
221
|
end
|
222
|
+
puts '# ---'
|
199
223
|
end
|
200
224
|
|
225
|
+
def update_stacks(stack_list, create_if_missing=false, synchronous=false, outputs=false)
|
226
|
+
stack_list.each do |stack|
|
227
|
+
res = update_stack(stack, create_if_missing, synchronous, outputs)
|
228
|
+
return res unless res
|
229
|
+
end
|
230
|
+
|
231
|
+
true
|
232
|
+
end
|
233
|
+
|
234
|
+
def update_all_stacks(create_if_missing=false, synchronous=false, outputs=false)
|
235
|
+
stacks = Dir.glob('stacks/*').map { |d| File.basename d }
|
236
|
+
update_stacks(stacks, create_if_missing, synchronous, outputs)
|
237
|
+
end
|
201
238
|
|
202
239
|
def clone_stack(stack, new_stack, force=false, commit=false)
|
203
240
|
if File.exist?(get_stack_directory(new_stack)) and force == false
|
@@ -216,7 +253,6 @@ end
|
|
216
253
|
|
217
254
|
def delete_stack(stack_name)
|
218
255
|
configure()
|
219
|
-
FileUtils.rm_rf(get_stack_directory(stack_name))
|
220
256
|
if @cfn.stacks[stack_name].exists?
|
221
257
|
puts "Deleting stack #{stack_name}"
|
222
258
|
@cfn.stacks[stack_name].delete
|
data/lib/clouds/version.rb
CHANGED
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: clouds
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.1.
|
4
|
+
version: 0.1.6
|
5
5
|
prerelease:
|
6
6
|
platform: ruby
|
7
7
|
authors:
|
@@ -9,7 +9,7 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date: 2014-
|
12
|
+
date: 2014-10-08 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: bundler
|