clouds 0.1.3 → 0.1.6

Sign up to get free protection for your applications and to get access to all the features.
Files changed (5) hide show
  1. data/README.md +23 -11
  2. data/bin/clouds +12 -4
  3. data/lib/clouds.rb +53 -17
  4. data/lib/clouds/version.rb +1 -1
  5. metadata +2 -2
data/README.md CHANGED
@@ -1,6 +1,11 @@
1
1
  # clouds
2
2
 
3
- This is a simple tool that aims to ease the handling of [AWS CloudFormation](https://aws.amazon.com/cloudformation/) stacks, following the [infrastructure as code](http://sdarchitect.wordpress.com/2012/12/13/infrastructure-as-code/) philosophy in ways that are not possible with the AWS console. Even though you can achieve pretty much the same using the AWS command line tools, `clouds` aims to be much easier to use, cleaner and more specialized in handling CloudFormation stacks as code.
3
+ This is a simple tool that aims to ease the handling of [AWS CloudFormation](https://aws.amazon.com/cloudformation/) stacks, following the [infrastructure as code](http://sdarchitect.wordpress.com/2012/12/13/infrastructure-as-code/) philosophy in ways that are not possible with the AWS console. Even though you can achieve pretty much the same things using the AWS command line tools, `clouds` aims to be much easier to use, cleaner and more specialized in handling CloudFormation stacks as code.
4
+
5
+ It was developed at [HERE](http://here.com) and we use it pretty much on a daily basis for updating our existing stacks and creating new ones.
6
+
7
+ For some details about the story behind it, you can see this [presentation](http://slidesha.re/U7SRPq) on Slideshare.
8
+
4
9
 
5
10
  ## Features
6
11
  - Upload templates and parameters from the current directory into AWS with a single intuitive command. They need to be located in stacks/stack_name/{template.json,parameters.yaml}
@@ -8,21 +13,28 @@ This is a simple tool that aims to ease the handling of [AWS CloudFormation](htt
8
13
  - Dump one or all of the stacks from your currently selected AWS account into the current directory (preferably stored in a version control system) under the 'stacks' directory, including both the template JSON code and the parameters that were used for launching it, which will then be saved in a human-friendly YAML file.
9
14
  - Perform updates on the AWS stacks once you modified the dumped data (template or parameters)
10
15
  - Validate templates for JSON correctness when uploading or updating a stack. The error messages should help you debug syntax errors.
11
- - Clone existing stacks to create new ones. It defaults to local clone, which then needs another update call to get in effect. This is an easy way to migrate infrastructure code and later to promote changes between environments, and can also be used easily with projects running in different AWS accounts, especially if using a simple AWS profile switcher shell alias like implemented on oh-my-zsh.
16
+ - Clone existing stacks to create new ones. It defaults to a local clone, which then needs another update call to get in effect. This is an easy way to migrate infrastructure 'code' around and makes it trivial to promote changes between environments, and can also be used easily with projects running in different AWS accounts. This is especially convenient together with an AWS profile switcher shell alias like implemented on the oh-my-zsh aws plugin.
12
17
  - Perform cost calculations when creating a new stack.
13
18
  - Delete existing stacks with a simple command
14
- - For authentication it can use AWS account credentials defined using either the AWS_ACCESS_KEY_ID&AWS_SECRET_ACCESS_KEY combination, or the AWS_DEFAULT_PROFILE environment variables
19
+ - For authentication it can use AWS account credentials defined using either the AWS_ACCESS_KEY_ID&AWS_SECRET_ACCESS_KEY combination, or the AWS_DEFAULT_PROFILE environment variable
15
20
  - Able to use temporary credentials, if the AWS_SECURITY_TOKEN variable is defined in the environment.
16
21
 
17
22
  ## Installation
18
23
 
24
+ ###On Ruby 1.9 or newer
25
+
19
26
  gem install clouds
20
27
 
21
- or using a [bundler](http://bundler.io/) Gemfile:
28
+ ###On Ruby 1.8.x
29
+
30
+ You might need to install `rubygems` separtely and you should be aware of the incompatibility between the more recent versions of `nokogiri`(a dependency of the `aws-sdk`, on which `clouds` indirectly depends on) and Ruby 1.8. This is pretty well-explained [here](http://ruby.awsblog.com/post/Tx2T9MFQJK7U74N/AWS-SDK-for-Ruby-and-Nokogiri).
22
31
 
23
- gem 'clouds'
32
+ You will need to manually install the latest compatible version of nokogiri and its build-time dependencies, as documented below:
24
33
 
25
- Note: On systems running Ruby 1.8.x such as RHEL5 and clones, you need to be aware of the incompatibility between the more recent versions of `nokogiri`(a dependency of the `aws-sdk`, that `clouds` indirectly depends on) and Ruby 1.8. This is pretty well-documented [here](http://ruby.awsblog.com/post/Tx2T9MFQJK7U74N/AWS-SDK-for-Ruby-and-Nokogiri). **You will need to manually install the latest compatible version of nokogiri and the AWS SDK for Ruby as documented on that page.**
34
+ apt-get install libxml2-dev libxslt-dev # Debian and clones
35
+ yum install libxml2-devel libxslt-devel # RedHat and clones
36
+ gem install nokogiri --version="<1.6"
37
+ gem install clouds
26
38
 
27
39
  ## Running
28
40
  Execute this in your shell:
@@ -53,7 +65,7 @@ Clone a stack (locally only, so you can perform some changes)
53
65
 
54
66
  clouds clone stack new_stack
55
67
 
56
- Upload the cloned stack to AWS CloudFormation
68
+ Upload the cloned stack to AWS CloudFormation. This can also be used against templates you downloaded from third other sources, not just for dumps.
57
69
 
58
70
  clouds update new_stack -c
59
71
 
@@ -65,7 +77,7 @@ Delete the new stack (needs --force)
65
77
  ## Development
66
78
 
67
79
  ### Build requirements
68
- - Ruby, rubygems and rake (some might already be there on Ruby > 1.9)
80
+ - Ruby, rubygems and rake (some of them might already be there on Ruby > 1.9)
69
81
  - Development headers for libxml2 and libxslt (-devel packages)
70
82
  - Bundler
71
83
 
@@ -83,15 +95,15 @@ Only in case you didn't do it before
83
95
  ### Updating the installed gem
84
96
 
85
97
  rake repackage
86
- sudo gem install pkg/clouds-0.1.0.gem
98
+ sudo gem install pkg/clouds-*.gem
87
99
 
88
100
  ### Build instructions
89
101
 
90
- Simply run (once you have the stated requirements satisfied)
102
+ Once you have the stated requirements satisfied, you can just package it as gem
91
103
 
92
104
  rake package
93
105
 
94
106
  ### Installing your own gem
95
107
 
96
- sudo gem install pkg/clouds-0.1.0.gem
108
+ sudo gem install pkg/clouds-*.gem
97
109
 
data/bin/clouds CHANGED
@@ -108,14 +108,22 @@ command :update do |c|
108
108
  c.switch [:a,:all], :negatable => false
109
109
  c.desc 'Create AWS stacks if the required sources exist locally'
110
110
  c.switch [:c,:create_missing], :negatable => false
111
+ c.desc 'Wait until finished (synchronous mode)'
112
+ c.switch [:w, :wait], :negatable => false
113
+ c.desc 'Print stack outputs in yaml format'
114
+ c.switch [:o, :outputs], :negatable => false
111
115
 
112
116
  c.action do |global_options,options,args|
113
117
  if args.empty? && options[:all]
114
- puts 'dumping all stacks'
115
- update_all_stacks(options[:create_missing])
118
+ puts '# updating all stacks'
119
+ raise 'Failed to update the stack' unless update_all_stacks(options[:create_missing],
120
+ options[:wait],
121
+ options[:outputs])
116
122
  elsif args.length >= 1
117
- puts "updating the following stacks: #{args.inspect}"
118
- update_stacks(args,options[:create_missing])
123
+ puts "# updating the following stacks: #{args.inspect}"
124
+ raise 'Failed to update the stack' unless update_stacks(args,options[:create_missing],
125
+ options[:wait],
126
+ options[:outputs])
119
127
  else # no arguments and --all flag
120
128
  raise 'This command needs a list of stacks, or the --all flag in order to update all the stacks.'
121
129
  end
data/lib/clouds.rb CHANGED
@@ -140,20 +140,29 @@ def dump_all_stacks(force=false)
140
140
  end
141
141
  end
142
142
 
143
- def update_stack(stack_name, create_if_missing=false)
144
- configure()
145
- stack=nil
143
+ def update_stack(stack_name, create_if_missing=false, synchronous=false, outputs=false)
144
+ configure
145
+ stack = nil
146
146
 
147
147
  template_content = read_file(get_template_path(stack_name))
148
148
  parameters_content = read_file(get_parameters_path(stack_name))
149
149
 
150
+ parameters_hash = {}
151
+
150
152
  begin
151
- parameters_hash = YAML.load(parameters_content)
153
+ yaml_hash = YAML.load(parameters_content)
152
154
  rescue => e
153
155
  puts e
154
156
  raise e
155
157
  end
156
158
 
159
+ yaml_hash.each do |k, v|
160
+ v = v.join(",") if v.is_a?(Array)
161
+ parameters_hash[k] = v
162
+ end
163
+
164
+ p parameters_hash
165
+
157
166
  raise 'Empty stack template' if template_content.nil? || template_content.empty?
158
167
 
159
168
  template_validation = @cfn.validate_template(template_content)
@@ -161,43 +170,71 @@ def update_stack(stack_name, create_if_missing=false)
161
170
 
162
171
  begin
163
172
  if @cfn.stacks[stack_name].exists?
164
- puts "Updating stack #{stack_name}"
173
+ puts "# Updating stack #{stack_name}"
165
174
  stack = @cfn.stacks[stack_name]
166
175
  stack_capabilities = stack.capabilities
167
176
  stack.update(:template => template_content,
168
177
  :parameters => parameters_hash,
169
178
  :capabilities => stack_capabilities)
179
+ status = wait_until_status(stack, %w( CREATE_IN_PROGRESS UPDATE_IN_PROGRESS )) if synchronous
170
180
  elsif create_if_missing
171
- puts "Creating stack #{stack_name}"
181
+ puts "# Creating stack #{stack_name}"
172
182
  stack = @cfn.stacks.create(stack_name,
173
183
  template_content,
174
184
  { :parameters => parameters_hash,
175
185
  :capabilities => ['CAPABILITY_IAM']})
186
+ status = wait_until_status(stack, %w( CREATE_IN_PROGRESS UPDATE_IN_PROGRESS )) if synchronous
176
187
  else
177
188
  puts "Skipping stack #{stack_name} since it's not defined in this AWS account, if the stack exists locally you might use the -c flag"
178
189
  end
179
- unless stack.nil?
180
- estimated_cost = stack.estimate_template_cost()
181
- puts "Estimated costs for the stack #{stack_name} is #{estimated_cost}"
190
+ if ! stack.nil? && %w( UPDATE_COMPLETE CREATE_COMPLETE ).include?(status)
191
+ estimated_cost = stack.estimate_template_cost
192
+ puts "# Estimated costs for the stack #{stack_name} is #{estimated_cost}"
193
+
194
+ print_stack_outputs(stack) if outputs
182
195
  end
183
196
  rescue => e
184
197
  puts e
185
198
  end
199
+
200
+ %w( UPDATE_COMPLETE CREATE_COMPLETE ).include? status
186
201
  end
187
202
 
188
- def update_stacks(stack_list, create_if_missing=false)
189
- stack_list.each do |stack|
190
- update_stack(stack, create_if_missing)
203
+ def wait_until_status(stack, status_arr)
204
+ while true
205
+ begin
206
+ status = stack.status
207
+ printf("# %s : %s\n", Time.now.strftime('%Y-%m-%d %H:%M:%S'), status)
208
+ return status unless status_arr.include? status
209
+ sleep 5
210
+ rescue => e
211
+ raise "Cannot retrieve stack status: #{e}"
212
+ end
191
213
  end
192
214
  end
193
215
 
194
- def update_all_stacks()
195
- Dir.foreach('stacks') do |stack|
196
- next if item == '.' or item == '..'
197
- update_stack(stack)
216
+ def print_stack_outputs(stack)
217
+ puts '# ---'
218
+ puts "# Outputs from #{stack.name}"
219
+ stack.outputs.each do |o|
220
+ puts "#{o.key}: #{o.value}"
198
221
  end
222
+ puts '# ---'
199
223
  end
200
224
 
225
+ def update_stacks(stack_list, create_if_missing=false, synchronous=false, outputs=false)
226
+ stack_list.each do |stack|
227
+ res = update_stack(stack, create_if_missing, synchronous, outputs)
228
+ return res unless res
229
+ end
230
+
231
+ true
232
+ end
233
+
234
+ def update_all_stacks(create_if_missing=false, synchronous=false, outputs=false)
235
+ stacks = Dir.glob('stacks/*').map { |d| File.basename d }
236
+ update_stacks(stacks, create_if_missing, synchronous, outputs)
237
+ end
201
238
 
202
239
  def clone_stack(stack, new_stack, force=false, commit=false)
203
240
  if File.exist?(get_stack_directory(new_stack)) and force == false
@@ -216,7 +253,6 @@ end
216
253
 
217
254
  def delete_stack(stack_name)
218
255
  configure()
219
- FileUtils.rm_rf(get_stack_directory(stack_name))
220
256
  if @cfn.stacks[stack_name].exists?
221
257
  puts "Deleting stack #{stack_name}"
222
258
  @cfn.stacks[stack_name].delete
@@ -1,3 +1,3 @@
1
1
  module Clouds
2
- VERSION = '0.1.3'
2
+ VERSION = '0.1.6'
3
3
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: clouds
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.3
4
+ version: 0.1.6
5
5
  prerelease:
6
6
  platform: ruby
7
7
  authors:
@@ -9,7 +9,7 @@ authors:
9
9
  autorequire:
10
10
  bindir: bin
11
11
  cert_chain: []
12
- date: 2014-05-28 00:00:00.000000000 Z
12
+ date: 2014-10-08 00:00:00.000000000 Z
13
13
  dependencies:
14
14
  - !ruby/object:Gem::Dependency
15
15
  name: bundler