convection 0.2.33 → 0.2.34.pre.beta.1

Sign up to get free protection for your applications and to get access to all the features.
Files changed (55) hide show
  1. checksums.yaml +4 -4
  2. data/.yardopts +1 -0
  3. data/CONTRIBUTING.md +22 -0
  4. data/README.md +15 -202
  5. data/Rakefile +3 -0
  6. data/docs/adding-new-resource-coverage.md +265 -0
  7. data/docs/canceling-stack-updates.md +5 -0
  8. data/docs/deleting-stacks.md +5 -0
  9. data/docs/getting-started.md +904 -0
  10. data/docs/index.md +69 -0
  11. data/docs/pygment.css +62 -0
  12. data/docs/relationship-to-cloudformation.md +51 -0
  13. data/docs/stacks.md +86 -0
  14. data/docs/template.html +130 -0
  15. data/example/getting-started-guide/Cloudfile +12 -0
  16. data/example/getting-started-guide/vpc.rb +74 -0
  17. data/example/stacks/Cloudfile +12 -0
  18. data/example/stacks/tasks/lookup_vpc_task.rb +28 -0
  19. data/example/stacks/templates/vpc.rb +14 -0
  20. data/lib/convection.rb +6 -0
  21. data/lib/convection/control/cloud.rb +1 -0
  22. data/lib/convection/control/stack.rb +126 -15
  23. data/lib/convection/model/cloudfile.rb +3 -0
  24. data/lib/convection/model/template/resource/aws_cloudfront_distribution.rb +24 -30
  25. data/lib/convection/model/template/resource/aws_ec2_dhcp_options.rb +38 -0
  26. data/lib/convection/model/template/resource/aws_ec2_security_group.rb +24 -2
  27. data/lib/convection/model/template/resource/aws_iam_user.rb +17 -3
  28. data/lib/convection/model/template/resource/aws_s3_bucket.rb +9 -3
  29. data/lib/convection/model/template/resource/aws_s3_bucket_policy.rb +10 -3
  30. data/lib/convection/model/template/resource/aws_sns_topic.rb +6 -3
  31. data/lib/convection/model/template/resource/aws_sns_topic_policy.rb +10 -3
  32. data/lib/convection/model/template/resource/aws_sqs_queue.rb +5 -3
  33. data/lib/convection/model/template/resource/aws_sqs_queue_policy.rb +10 -3
  34. data/spec/convection/model/template/resource/ec2_dhcp_options_spec.rb +55 -0
  35. data/yard_extensions.rb +4 -0
  36. data/yard_extensions/properties_handler.rb +30 -0
  37. data/yard_extensions/type_handler.rb +188 -0
  38. metadata +27 -23
  39. data/example/Cloudfile +0 -13
  40. data/example/deprecated/elb.rb +0 -27
  41. data/example/deprecated/iam_access_key.rb +0 -18
  42. data/example/deprecated/iam_group.rb +0 -31
  43. data/example/deprecated/iam_role.rb +0 -52
  44. data/example/deprecated/iam_user.rb +0 -31
  45. data/example/deprecated/rds.rb +0 -70
  46. data/example/deprecated/s3.rb +0 -13
  47. data/example/deprecated/sqs.rb +0 -32
  48. data/example/deprecated/vpc.rb +0 -85
  49. data/example/instances.rb +0 -93
  50. data/example/output/vpc.json +0 -335
  51. data/example/security-groups.rb +0 -77
  52. data/example/sqs-queue/Cloudfile +0 -19
  53. data/example/sqs-queue/README.md +0 -12
  54. data/example/trust_cloudtrail.rb +0 -24
  55. data/example/vpc.rb +0 -143
@@ -0,0 +1,5 @@
1
+ # Canceling Stack Updates
2
+ To cancel a convection converge
3
+
4
+ 1. Kill the converge with control + c.
5
+ 2. Following the directions provided here to cancel the update and trigger a rollback. http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn--stack-update-cancel.html
@@ -0,0 +1,5 @@
1
+ # Deleting Convection Stacks
2
+ To delete a convection stack
3
+
4
+ 1. Delete the stack from the Cloudfile and converge.
5
+ 2. Follow the directions provided here to delete it from CloudFormation. http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-console-delete-stack.html
@@ -0,0 +1,904 @@
1
+ # Getting Started #
2
+
3
+ This guide walks you through the creation of a Convection template that matches
4
+ up with one of [Amazon's VPC Wizard Scenarios][vpc2], creating a VPC with public
5
+ and private subnets.
6
+
7
+ By the end of this guide you'll have the following resources in your Amazon
8
+ account
9
+
10
+ * A CloudFormation template describing everything in this guide
11
+ * A VPC with two subnets, one public, one private
12
+ * A NAT router so EC2 instances in the private subnet can reach the internet
13
+ * A security group for the NAT router so you can control access to it
14
+
15
+ To get started, create the following directory structure for your project. If your region is not us-east-1 then change that to your region. If you have multiple regions create multiple region folders each with their own cloud file.
16
+ ```
17
+ my-convection-project/
18
+ ├── clouds
19
+ │   └── us-east-1
20
+ └── templates
21
+ ```
22
+ In the top level of your convection project create a file "Gemfile" with the following inside.
23
+ ```ruby
24
+ gem 'convection'
25
+ ```
26
+
27
+ In the us-east-1 folder open a new file named "Cloudfile" and put the following Ruby
28
+ ```ruby
29
+ Dir.glob('./../../templates/**.rb') do |file|
30
+ require_relative file
31
+ end
32
+
33
+ region 'us-east-1'
34
+ name 'convection-demo'
35
+ ```
36
+
37
+ Cloudfiles are written using Convection's DSL. The "convection" gem defines
38
+ methods like `region` and `name`. The `region` method tells Convection where
39
+ AWS resources should be created. The `name` method provides a common identifier
40
+ for grouping AWS resources.
41
+
42
+ The core of Convection's DSL consists of two parts, templates and stacks.
43
+ Templates let you to describe AWS resources in a reusable fashion. Stacks are
44
+ instances of a template in Amazon. We're going to start by creating a template
45
+ and stack to define our VPC.
46
+
47
+ Update your Cloudfile to look like the one below.
48
+
49
+ ```ruby
50
+ Dir.glob('./../../templates/**.rb') do |file|
51
+ require_relative file
52
+ end
53
+
54
+ region 'us-east-1'
55
+ name 'convection-demo'
56
+
57
+ stack 'vpc', Templates::VPC
58
+ ```
59
+ In the templates directory create a vpc.rb file and include the following in it.
60
+ ```ruby
61
+ require 'convection'
62
+
63
+ module Templates
64
+ VPC = Convection.template do
65
+ description 'VPC with Public and Private Subnets (NAT)'
66
+
67
+ end
68
+ end
69
+ ```
70
+
71
+ The `template` method creates a Convection template. The `description` property
72
+ in the template maps to the "Description" property in a [CloudFormation template][cf-template].
73
+ Assigning the result of the `template` method to the `VPC` variable allows us to
74
+ pass the template to the `stack` method. The `stack` method defines a
75
+ CloudFormation stack in AWS.
76
+
77
+ This separation between the description of AWS resources (the template) and
78
+ the instantiation of those resources (the stack) makes Convection templates
79
+ flexible.
80
+
81
+ Now that we've got a Convection template, we can run Convection's diff command
82
+ to compare what's in our template with what's in Amazon. This is a new template,
83
+ so we should only see new resources being created.
84
+
85
+ ```text
86
+ $> convection diff
87
+
88
+ compare Compare local state of stack vpc (convection-demo-vpc) with remote template
89
+ create .AWSTemplateFormatVersion: 2010-09-09
90
+ create .Description: VPC with Public and Private Subnets (NAT)
91
+ ```
92
+
93
+ The first line of output tells us that Convection's comparing our "vpc" stack
94
+ with the remote CloudFormation stack in Amazon. Since this is a new stack,
95
+ Convection prints two more lines telling us it's going to create the stack and
96
+ give it a description.
97
+
98
+ Now that we know what Convection's going to do, we can ask it to check that
99
+ stack creation will succeed. To do that, we run Convection's validate command.
100
+
101
+ ```text
102
+ $> convection validate vpc
103
+
104
+ Template format error: At least one Resources member must be defined.
105
+ ```
106
+
107
+ The validate command takes a single argument, the name of the stack to validate.
108
+ We only have the "vpc" stack, so that's what we're validating. Convection
109
+ tells us that our template's not valid. We're missing a resource.
110
+
111
+ Since we're building a VPC with two subnets, let's add the VPC itself as a
112
+ resource. Our VPC will be of size /23 and use the CIDR block 10.10.10.0/23.
113
+
114
+ Update your vpc.rb template to look like the one below.
115
+
116
+ ```ruby
117
+ require 'convection'
118
+
119
+ module Templates
120
+ VPC = Convection.template do
121
+ description 'VPC with Public and Private Subnets (NAT)'
122
+
123
+ ec2_vpc 'DemoVPC' do
124
+ network '10.10.10.0/23'
125
+ end
126
+ end
127
+ end
128
+ ```
129
+
130
+ The `ec2_vpc` method defines an [AWS::EC2::VPC][cf-vpc] resource in
131
+ Convection. VPC resources require a CIDR block. We set the `network`
132
+ property in our template to the CIDR block we want our VPC to use. Now
133
+ we can validate our template.
134
+
135
+ ```text
136
+ $> convection validate vpc
137
+
138
+ Template validated successfully
139
+ ```
140
+
141
+ Our template's good, so we'll diff it again to see what's going to change.
142
+
143
+ ```text
144
+ $> convection diff
145
+
146
+ compare Compare local state of stack vpc (convection-demo-vpc) with remote template
147
+ create .AWSTemplateFormatVersion: 2010-09-09
148
+ create .Description: VPC with Public and Private Subnets (NAT)
149
+ create .Resources.DemoVPC.Type: AWS::EC2::VPC
150
+ create .Resources.DemoVPC.Properties.CidrBlock: 10.10.10.0/23
151
+ ```
152
+
153
+ That looks correct. Convection will create a VPC with a CIDR block of
154
+ 10.10.10.0/23. We can run Convection's converge command to create the VPC in
155
+ Amazon.
156
+
157
+ ```text
158
+ $> convection converge
159
+
160
+ converge Stack vpc
161
+ create_in_progress convection-demo-vpc: (AWS::CloudFormation::Stack) User Initiated
162
+ create_in_progress DemoVPC: (AWS::EC2::VPC)
163
+ create_in_progress DemoVPC: (AWS::EC2::VPC) Resource creation Initiated
164
+ create_complete DemoVPC: (AWS::EC2::VPC)
165
+ create_complete convection-demo-vpc: (AWS::CloudFormation::Stack)
166
+ ```
167
+
168
+ If we look at the Amazon web console, you can see Convection's created two
169
+ new things for us. There's a CloudFormation stack named "convection-demo-vpc"
170
+ and there's the actual VPC resource, which is currently unnamed.
171
+
172
+ We can name the VPC resource by tagging it. Setting the `tag` attribute on
173
+ the "DemoVPC" resource in Convection will add a tag to the resource. We'll use
174
+ a combination of the cloud name and the stack name as the tag for the VPC.
175
+
176
+ Update your vpc.rb template to look like the one below.
177
+
178
+ ```ruby
179
+ require 'convection'
180
+
181
+ module Templates
182
+ VPC = Convection.template do
183
+ description 'VPC with Public and Private Subnets (NAT)'
184
+
185
+ ec2_vpc 'DemoVPC' do
186
+ network '10.10.10.0/23'
187
+ tag 'Name', "#{stack.cloud}-#{stack.name}"
188
+ end
189
+ end
190
+ end
191
+ ```
192
+
193
+ We're using a naming convection for resources that includes the cloud name and
194
+ the stack name. That makes it easy to look through the Amazon web console and
195
+ see which resources belong to which Convection managed clouds. Now we run
196
+ the diff command to see the changes Convection will apply.
197
+
198
+ ```text
199
+ $> convection diff
200
+
201
+ compare Compare local state of stack vpc (convection-demo-vpc) with remote template
202
+ create .Resources.DemoVPC.Properties.Tags.0.Key: Name
203
+ create .Resources.DemoVPC.Properties.Tags.0.Value: convection-demo-vpc
204
+ ```
205
+ To see what the cloud formation template for your vpc template would look like you can run `convection print vpc`.
206
+ This can help you verify that values referenced under the `stack` namespace are set correctly.
207
+
208
+ ```json
209
+ $> convection print vpc
210
+ {
211
+ "AWSTemplateFormatVersion": "2010-09-09",
212
+ "Description": "VPC with Public and Private Subnets (NAT)",
213
+ "Parameters": {
214
+ },
215
+ "Mappings": {
216
+ },
217
+ "Conditions": {
218
+ },
219
+ "Resources": {
220
+ "DemoVPC": {
221
+ "Type": "AWS::EC2::VPC",
222
+ "Properties": {
223
+ "CidrBlock": "10.10.10.0/23",
224
+ "Tags": [
225
+ {
226
+ "Key": "Name",
227
+ "Value": "convection-demo-vpc"
228
+ }
229
+ ]
230
+ }
231
+ }
232
+ },
233
+ "Outputs": {
234
+ }
235
+ }
236
+ ```
237
+
238
+ Convection's going to add the "Name" tag to the "DemoVPC" resource. That's what
239
+ we want, so we can converge our template.
240
+
241
+ ```text
242
+ $> convection converge
243
+
244
+ converge Stack vpc
245
+ update_in_progress convection-demo-vpc: (AWS::CloudFormation::Stack) User Initiated
246
+ update_in_progress DemoVPC: (AWS::EC2::VPC)
247
+ update_complete DemoVPC: (AWS::EC2::VPC)
248
+ update_complete_cleanup_in_progress convection-demo-vpc: (AWS::CloudFormation::Stack)
249
+ update_complete convection-demo-vpc: (AWS::CloudFormation::Stack)
250
+ ```
251
+
252
+ Looking at the Amazon web console, we can see that our VPC is now named
253
+ "convection-demo-vpc".
254
+
255
+ ## Creating a private subnet ##
256
+
257
+ The CIDR block for our VPC is 10.10.10/23. We'll set up a public subnet
258
+ with a CIDR block of 10.10.11.0/24 and a private subnet with a CIDR block of
259
+ 10.10.10.0/24. Let's create the private subnet first.
260
+
261
+ Just like there's an `ec2_vpc` method in Convection for creating a VPC, there's
262
+ also an `ec2_subnet` method for creating a subnet. And like our VPC, our subnet
263
+ will have `network` and `tag` attributes.
264
+
265
+ Update your vpc.rb template to look like the one below.
266
+
267
+ ```ruby
268
+ require 'convection'
269
+
270
+ module Templates
271
+ VPC = Convection.template do
272
+ description 'VPC with Public and Private Subnets (NAT)'
273
+
274
+ ec2_vpc 'DemoVPC' do
275
+ network '10.10.10.0/23'
276
+ tag 'Name', "#{stack.cloud}-#{stack.name}"
277
+ end
278
+
279
+ ec2_subnet 'PrivateSubnet' do
280
+ network '10.10.10.0/24'
281
+ tag 'Name', "#{stack.cloud}-#{stack.name}-private"
282
+ end
283
+
284
+ end
285
+ end
286
+ ```
287
+
288
+ We can run the diff command to see what Convection's going to create.
289
+
290
+ ```text
291
+ $> convection diff
292
+
293
+ compare Compare local state of stack vpc (convection-demo-vpc) with remote template
294
+ create .Resources.PrivateSubnet.Type: AWS::EC2::Subnet
295
+ create .Resources.PrivateSubnet.Properties.CidrBlock: 10.10.10.0/24
296
+ create .Resources.PrivateSubnet.Properties.Tags.0.Key: Name
297
+ create .Resources.PrivateSubnet.Properties.Tags.0.Value: convection-demo-vpc-private
298
+ ```
299
+
300
+ There's our new private subnet with its CIDR block and "Name" tag. If we look at
301
+ the documentation for the [AWS::EC2::Subnet][cf-subnet] resource, we
302
+ can see it has a required "VpcId" attribute. We can use the Convection's
303
+ `fn_ref` method to get the logical ID of our VPC resource and pass it in to the subnet.
304
+
305
+ Update your vpc.rb template to look like the one below.
306
+
307
+ ```ruby
308
+ require 'convection'
309
+
310
+ module Templates
311
+ VPC = Convection.template do
312
+ description 'VPC with Public and Private Subnets (NAT)'
313
+
314
+ ec2_vpc 'DemoVPC' do
315
+ network '10.10.10.0/23'
316
+ tag 'Name', "#{stack.cloud}-#{stack.name}"
317
+ end
318
+
319
+ ec2_subnet 'PrivateSubnet' do
320
+ network '10.10.10.0/24'
321
+ tag 'Name', "#{stack.cloud}-#{stack.name}-private"
322
+ vpc fn_ref('DemoVPC')
323
+ end
324
+
325
+ end
326
+ end
327
+ ```
328
+
329
+ Now we can diff the Cloudfile and see that the "VpcId" property's getting set
330
+ on our subnet.
331
+
332
+ ```text
333
+ $> convection diff
334
+
335
+ compare Compare local state of stack vpc (convection-demo-vpc) with remote template
336
+ create .Resources.PrivateSubnet.Type: AWS::EC2::Subnet
337
+ create .Resources.PrivateSubnet.Properties.VpcId.Ref: DemoVPC
338
+ create .Resources.PrivateSubnet.Properties.CidrBlock: 10.10.10.0/24
339
+ create .Resources.PrivateSubnet.Properties.Tags.0.Key: Name
340
+ create .Resources.PrivateSubnet.Properties.Tags.0.Value: convection-demo-vpc-private
341
+ ```
342
+
343
+ Let's converge the Cloudfile, and create a new private subnet in Amazon.
344
+
345
+ ```text
346
+ $> convection converge
347
+
348
+ converge Stack vpc
349
+ update_in_progress convection-demo-vpc: (AWS::CloudFormation::Stack) User Initiated
350
+ create_in_progress PrivateSubnet: (AWS::EC2::Subnet)
351
+ create_in_progress PrivateSubnet: (AWS::EC2::Subnet) Resource creation Initiated
352
+ create_complete PrivateSubnet: (AWS::EC2::Subnet)
353
+ update_complete_cleanup_in_progress convection-demo-vpc: (AWS::CloudFormation::Stack)
354
+ update_complete convection-demo-vpc: (AWS::CloudFormation::Stack)
355
+ ```
356
+
357
+ ## Creating a public subnet ##
358
+
359
+ Creating a public subnet is exactly the same as creating a private subnet.
360
+ Use the `ec2_subnet` method to create an [AWS::EC2::Subnet][cf-subnet]
361
+ resource and give it a CIDR block of 10.10.11.0/24. Insert the following convection code into your vpc.rb template to accomplish this.
362
+
363
+ ```ruby
364
+ ec2_subnet 'PublicSubnet' do
365
+ network '10.10.11.0/24'
366
+ tag 'Name', "#{stack.cloud}-#{stack.name}-public"
367
+ vpc fn_ref('DemoVPC')
368
+ end
369
+ ```
370
+
371
+ Looking at the documentation for the [AWS::EC2::Subnet][cf-subnet]
372
+ resource, the major difference between a public and private subnet is
373
+ that Amazon assigns IP addresses to public subnets. Making a subnet public is
374
+ a matter of setting the "MapPublicIpOnLaunch" property to `true`.
375
+
376
+ Add the below line to your `PublicSubnet` block
377
+
378
+ ```ruby
379
+ public_ips true
380
+ ```
381
+
382
+ Your diff should contain your changes for the public subnet.
383
+
384
+ ```text
385
+ $> convection diff
386
+
387
+ compare Compare local state of stack vpc (convection-demo-vpc) with remote template
388
+ create .Resources.PublicSubnet.Type: AWS::EC2::Subnet
389
+ create .Resources.PublicSubnet.Properties.VpcId.Ref: DemoVPC
390
+ create .Resources.PublicSubnet.Properties.CidrBlock: 10.10.11.0/24
391
+ create .Resources.PublicSubnet.Properties.MapPublicIpOnLaunch: true
392
+ create .Resources.PublicSubnet.Properties.Tags.0.Key: Name
393
+ create .Resources.PublicSubnet.Properties.Tags.0.Value: convection-demo-vpc-public
394
+ ```
395
+
396
+ Convection says it will create our public subnet with the "MapPublicIpOnLaunch"
397
+ property set to `true`. It also doesn't show any changes to our private subnet.
398
+ That's what we expect, so we can converge our template.
399
+
400
+ ```text
401
+ $> convection converge
402
+
403
+ converge Stack vpc
404
+ update_in_progress convection-demo-vpc: (AWS::CloudFormation::Stack) User Initiated
405
+ create_in_progress PublicSubnet: (AWS::EC2::Subnet)
406
+ create_in_progress PublicSubnet: (AWS::EC2::Subnet) Resource creation Initiated
407
+ create_complete PublicSubnet: (AWS::EC2::Subnet)
408
+ update_complete_cleanup_in_progress convection-demo-vpc: (AWS::CloudFormation::Stack)
409
+ update_complete convection-demo-vpc: (AWS::CloudFormation::Stack)
410
+ ```
411
+
412
+ If we look at our subnets in the Amazon web console, we can see the public
413
+ subnet has the "Auto-assign Public IP" attribute set to "yes" and the private
414
+ subnet has that value set to "no". Any EC2 instances we create in the public
415
+ subnet will automatically get public IP addresses.
416
+
417
+ ## Create a security group for the NAT router ##
418
+
419
+ Our NAT router is going to live in the public subnet. However, we want to
420
+ restrict access so only EC2 instances in our private subnet can use it. We can
421
+ do this with a security group.
422
+
423
+ Convection provides an `ec2_security_group` method for creating security groups.
424
+ The [AWS::EC2::SecurityGroup][cf-security-group] resource requires a description
425
+ and a reference to our VPC. We can add the `ec2_security_group` method to our
426
+ Cloudfile with a `description` and `vpc` attribute to create a default security
427
+ group for our NAT router.
428
+
429
+ Add the below block to your vpc.rb template
430
+
431
+ ```ruby
432
+ ec2_security_group 'NATSecurityGroup' do
433
+ description 'NAT access for private subnet'
434
+ vpc fn_ref('DemoVPC')
435
+ tag 'Name', "#{stack.cloud}-#{stack.name}-nat-security-group"
436
+ end
437
+ ```
438
+
439
+ We'll follow the same pattern we've used before. Diff the Convection template to
440
+ make sure it does what we expect, then converge it. This pattern of diffing then
441
+ converging is useful when paired with code reviews. You can make changes to a
442
+ template, diff it, then have the changes and diff reviewed before you converge.
443
+
444
+ ```text
445
+ $> convection diff
446
+
447
+ compare Compare local state of stack vpc (convection-demo-vpc) with remote template
448
+ create .Resources.NATSecurityGroup.Type: AWS::EC2::SecurityGroup
449
+ create .Resources.NATSecurityGroup.Properties.GroupDescription: NAT access for private subnet
450
+ create .Resources.NATSecurityGroup.Properties.VpcId.Ref: DemoVPC
451
+ create .Resources.NATSecurityGroup.Properties.Tags.0.Key: Name
452
+ create .Resources.NATSecurityGroup.Properties.Tags.0.Value: convection-demo-vpc-nat-security-group
453
+ ```
454
+
455
+ ```text
456
+ $> convection converge
457
+
458
+ converge Stack vpc
459
+ update_in_progress convection-demo-vpc: (AWS::CloudFormation::Stack) User Initiated
460
+ create_in_progress NATSecurityGroup: (AWS::EC2::SecurityGroup)
461
+ create_in_progress NATSecurityGroup: (AWS::EC2::SecurityGroup) Resource creation Initiated
462
+ create_complete NATSecurityGroup: (AWS::EC2::SecurityGroup)
463
+ update_complete_cleanup_in_progress convection-demo-vpc: (AWS::CloudFormation::Stack)
464
+ update_complete convection-demo-vpc: (AWS::CloudFormation::Stack)
465
+ ```
466
+
467
+ Looking at our new security group in the Amazon web console, we can see it
468
+ doesn't have any inbound rules. Lets update our security group to allow inbound
469
+ web traffic from our private subnet.
470
+
471
+ Within the `ec2_security_group` method, Convection provides the `ingress_rule`
472
+ helper method for defining inbound rules. The method takes a traffic type, a
473
+ port number, and a block for setting the rule's `source` attribute.
474
+
475
+ Update the block in your `ec2_security_group` method to look like the one below.
476
+
477
+ ```ruby
478
+ ec2_security_group 'NATSecurityGroup' do
479
+ description 'NAT access for private subnet'
480
+ vpc fn_ref('DemoVPC')
481
+ tag 'Name', "#{stack.cloud}-#{stack.name}-nat-security-group"
482
+ ingress_rule :tcp, 443 do
483
+ source '10.10.10.0/24'
484
+ end
485
+ ingress_rule :tcp, 80 do
486
+ source '10.10.10.0/24'
487
+ end
488
+ end
489
+ ```
490
+
491
+ This locks down our NAT router so it can only receive requests for web traffic
492
+ from our private subnet. Having ingress rules for ports 443 and 80 allows us to
493
+ handle both HTTPS and HTTP traffic.
494
+
495
+ By default, Amazon sets an outbound rule on our security group that allows all
496
+ traffic. Since our NAT router only handles web traffic, we can add egress rules
497
+ as well and lock down outbound requests. Convection has an `egress_rule` method
498
+ for setting output traffic rules. It has the same syntax as the `ingress_rule`
499
+ method.
500
+
501
+ Your vpc.rb template should now look like the one below.
502
+
503
+ ```ruby
504
+ require 'convection'
505
+
506
+ module Templates
507
+ VPC = Convection.template do
508
+ description 'VPC with Public and Private Subnets (NAT)'
509
+
510
+ ec2_vpc 'DemoVPC' do
511
+ network '10.10.10.0/23'
512
+ tag 'Name', "#{stack.cloud}-#{stack.name}"
513
+ end
514
+
515
+ ec2_subnet 'PrivateSubnet' do
516
+ network '10.10.10.0/24'
517
+ tag 'Name', "#{stack.cloud}-#{stack.name}-private"
518
+ vpc fn_ref('DemoVPC')
519
+ end
520
+
521
+ ec2_subnet 'PublicSubnet' do
522
+ network '10.10.11.0/24'
523
+ tag 'Name', "#{stack.cloud}-#{stack.name}-public"
524
+ vpc fn_ref('DemoVPC')
525
+ public_ips true
526
+ end
527
+
528
+ ec2_security_group 'NATSecurityGroup' do
529
+ description 'NAT access for private subnet'
530
+ vpc fn_ref('DemoVPC')
531
+ tag 'Name', "#{stack.cloud}-#{stack.name}-nat-security-group"
532
+ ingress_rule :tcp, 443 do
533
+ source '10.10.10.0/24'
534
+ end
535
+ ingress_rule :tcp, 80 do
536
+ source '10.10.10.0/24'
537
+ end
538
+ egress_rule :tcp, 443 do
539
+ source '0.0.0.0/0'
540
+ end
541
+ egress_rule :tcp, 80 do
542
+ source '0.0.0.0/0'
543
+ end
544
+ end
545
+
546
+ end
547
+ end
548
+ ```
549
+
550
+ Our security group's locked down, so we can diff our template and see what
551
+ changes.
552
+
553
+ ```text
554
+ $> convection diff
555
+
556
+ compare Compare local state of stack vpc (convection-demo-vpc) with remote template
557
+ create .Resources.NATSecurityGroup.Properties.SecurityGroupIngress.0.IpProtocol: 6
558
+ create .Resources.NATSecurityGroup.Properties.SecurityGroupIngress.0.FromPort: 443
559
+ create .Resources.NATSecurityGroup.Properties.SecurityGroupIngress.0.ToPort: 443
560
+ create .Resources.NATSecurityGroup.Properties.SecurityGroupIngress.0.CidrIp: 10.10.10.0/24
561
+ create .Resources.NATSecurityGroup.Properties.SecurityGroupIngress.1.IpProtocol: 6
562
+ create .Resources.NATSecurityGroup.Properties.SecurityGroupIngress.1.FromPort: 80
563
+ create .Resources.NATSecurityGroup.Properties.SecurityGroupIngress.1.ToPort: 80
564
+ create .Resources.NATSecurityGroup.Properties.SecurityGroupIngress.1.CidrIp: 10.10.10.0/24
565
+ create .Resources.NATSecurityGroup.Properties.SecurityGroupEgress.0.IpProtocol: 6
566
+ create .Resources.NATSecurityGroup.Properties.SecurityGroupEgress.0.FromPort: 443
567
+ create .Resources.NATSecurityGroup.Properties.SecurityGroupEgress.0.ToPort: 443
568
+ create .Resources.NATSecurityGroup.Properties.SecurityGroupEgress.0.CidrIp: 0.0.0.0/0
569
+ create .Resources.NATSecurityGroup.Properties.SecurityGroupEgress.1.IpProtocol: 6
570
+ create .Resources.NATSecurityGroup.Properties.SecurityGroupEgress.1.FromPort: 80
571
+ create .Resources.NATSecurityGroup.Properties.SecurityGroupEgress.1.ToPort: 80
572
+ create .Resources.NATSecurityGroup.Properties.SecurityGroupEgress.1.CidrIp: 0.0.0.0/0
573
+ ```
574
+
575
+ It's exactly what we expect. Looks like it's just the ingress and egress rules,
576
+ so we can converge the template and apply the changes.
577
+
578
+ ```text
579
+ $> convection converge
580
+
581
+ converge Stack vpc
582
+ update_in_progress convection-demo-vpc: (AWS::CloudFormation::Stack) User Initiated
583
+ update_in_progress NATSecurityGroup: (AWS::EC2::SecurityGroup)
584
+ update_complete NATSecurityGroup: (AWS::EC2::SecurityGroup)
585
+ update_complete_cleanup_in_progress convection-demo-vpc: (AWS::CloudFormation::Stack)
586
+ update_complete convection-demo-vpc: (AWS::CloudFormation::Stack)
587
+ ```
588
+
589
+ ## Create the NAT router ##
590
+
591
+ Now that we've got our security group, we need to set up an EC2 instance that
592
+ will function as a NAT router. We'll be using one of [Amazon's pre-built NAT
593
+ images][nat-instance] since we don't need anything custom. Open up the Amazon
594
+ web console and search for AMIs with the string "amzn-ami-vpc-nat-pv" in their
595
+ name. The most recent one, from March 2015, has ID ami-c02b04a8.
596
+
597
+ The [AWS::EC2::Instance][cf-ec2] resource handles creating our
598
+ router instance from the given AMI. Since our router provides internet access,
599
+ it needs to be in the public subnet. We also need to disable source/destination
600
+ checking so it can perform network address translation. Finally, we'll make
601
+ sure the instance is in the security group we just created.
602
+
603
+ Update your vpc.rb template to look like the one below.
604
+
605
+ ```ruby
606
+ require 'convection'
607
+
608
+ module Templates
609
+ VPC = Convection.template do
610
+ description 'VPC with Public and Private Subnets (NAT)'
611
+
612
+ ec2_vpc 'DemoVPC' do
613
+ network '10.10.10.0/23'
614
+ tag 'Name', "#{stack.cloud}-#{stack.name}"
615
+ end
616
+
617
+ ec2_subnet 'PrivateSubnet' do
618
+ network '10.10.10.0/24'
619
+ tag 'Name', "#{stack.cloud}-#{stack.name}-private"
620
+ vpc fn_ref('DemoVPC')
621
+ end
622
+
623
+ ec2_subnet 'PublicSubnet' do
624
+ network '10.10.11.0/24'
625
+ tag 'Name', "#{stack.cloud}-#{stack.name}-public"
626
+ vpc fn_ref('DemoVPC')
627
+ public_ips true
628
+ end
629
+
630
+ ec2_security_group 'NATSecurityGroup' do
631
+ description 'NAT access for private subnet'
632
+ vpc fn_ref('DemoVPC')
633
+ tag 'Name', "#{stack.cloud}-#{stack.name}-nat-security-group"
634
+ ingress_rule :tcp, 443 do
635
+ source '10.10.10.0/24'
636
+ end
637
+ ingress_rule :tcp, 80 do
638
+ source '10.10.10.0/24'
639
+ end
640
+ egress_rule :tcp, 443 do
641
+ source '0.0.0.0/0'
642
+ end
643
+ egress_rule :tcp, 80 do
644
+ source '0.0.0.0/0'
645
+ end
646
+ end
647
+
648
+ ec2_instance 'NATInstance' do
649
+ tag 'Name', "#{stack.cloud}-#{stack.name}-nat"
650
+ image_id 'ami-c02b04a8'
651
+ subnet fn_ref('PublicSubnet')
652
+ security_group fn_ref('NATSecurityGroup')
653
+ src_dst_checks false
654
+ end
655
+
656
+ end
657
+ end
658
+ ```
659
+
660
+ We can diff our template to see what's going to change.
661
+
662
+ ```text
663
+ $> convection diff
664
+
665
+ compare Compare local state of stack vpc (convection-demo-vpc) with remote template
666
+ create .Resources.NATInstance.Type: AWS::EC2::Instance
667
+ create .Resources.NATInstance.Properties.ImageId: ami-c02b04a8
668
+ create .Resources.NATInstance.Properties.SubnetId.Ref: PublicSubnet
669
+ create .Resources.NATInstance.Properties.SecurityGroupIds.0.Ref: NATSecurityGroup
670
+ delete .Resources.NATInstance.Properties.SourceDestCheck
671
+ create .Resources.NATInstance.Properties.Tags.0.Key: Name
672
+ create .Resources.NATInstance.Properties.Tags.0.Value: convection-demo-vpc-nat
673
+ ```
674
+
675
+ It's just our NAT router that's new, so let's converge it.
676
+
677
+ ```text
678
+ $> convection converge
679
+
680
+ converge Stack vpc
681
+ update_in_progress convection-demo-vpc: (AWS::CloudFormation::Stack) User Initiated
682
+ create_in_progress NATInstance: (AWS::EC2::Instance)
683
+ create_in_progress NATInstance: (AWS::EC2::Instance) Resource creation Initiated
684
+ create_complete NATInstance: (AWS::EC2::Instance)
685
+ update_complete_cleanup_in_progress convection-demo-vpc: (AWS::CloudFormation::Stack)
686
+ update_complete convection-demo-vpc: (AWS::CloudFormation::Stack)
687
+ ```
688
+ ## Create a route table for the public subnet ##
689
+
690
+ In order for instances in our public subnet to reach the internet, our VPC needs
691
+ an [AWS::EC2::InternetGateway][cf-internet-gateway] resource. Using stock
692
+ CloudFormation, we'd create the gateway and wire it up to to our VPC with an
693
+ [AWS::EC2::VPCGatewayAttachment][cf-gateway-attachment]. We'd also need a with
694
+ [AWS::EC2::RouteTable][cf-route-table] resource for the gateway with a
695
+ default [AWS::EC2::Route][cf-route] resource that lets it connect to the
696
+ world.
697
+
698
+ We could use Convection to create each of those resources. However, there's a
699
+ simpler way. Convection provides an `add_route_table` method that can generate
700
+ an internet gateway and wire it up to our VPC.
701
+
702
+ Update your `ec2_vpc` block to look like the one below. NOTE we added `enable_dns` and `add_route_table`.
703
+ ```ruby
704
+ ec2_vpc 'DemoVPC' do
705
+ network '10.10.10.0/23'
706
+ tag 'Name', "#{stack.cloud}-#{stack.name}"
707
+ enable_dns true
708
+ add_route_table 'InternetGateway', gateway_route: true
709
+ end
710
+ ```
711
+
712
+ Diffing our template, we can see our VPC will get a Route, RouteTable,
713
+ InternetGateway, and InternetGateway attachment. The route lets the internet
714
+ gateway talk to the world, which is exactly what we want. Notice that we're
715
+ also enabling DNS support and hostnames on our VPC.
716
+
717
+ ```text
718
+ $> convection diff
719
+
720
+ compare Compare local state of stack vpc (convection-demo-vpc) with remote template
721
+ create .Resources.DemoVPCTableInternetGateway.Type: AWS::EC2::RouteTable
722
+ create .Resources.DemoVPCTableInternetGateway.Properties.VpcId.Ref: DemoVPC
723
+ create .Resources.DemoVPCTableInternetGateway.Properties.Tags.0.Key: Name
724
+ create .Resources.DemoVPCTableInternetGateway.Properties.Tags.0.Value: DemoVPCTableInternetGateway
725
+ create .Resources.DemoVPCIGVPCAttachmentDemoVPC.Type: AWS::EC2::VPCGatewayAttachment
726
+ create .Resources.DemoVPCIGVPCAttachmentDemoVPC.Properties.VpcId.Ref: DemoVPC
727
+ create .Resources.DemoVPCIGVPCAttachmentDemoVPC.Properties.InternetGatewayId.Ref: DemoVPCIG
728
+ create .Resources.DemoVPCIG.Type: AWS::EC2::InternetGateway
729
+ create .Resources.DemoVPCIG.Properties.Tags.0.Key: Name
730
+ create .Resources.DemoVPCIG.Properties.Tags.0.Value: DemoVPCInternetGateway
731
+ create .Resources.DemoVPCTableInternetGatewayRouteDefault.Type: AWS::EC2::Route
732
+ create .Resources.DemoVPCTableInternetGatewayRouteDefault.Properties.RouteTableId.Ref: DemoVPCTableInternetGateway
733
+ create .Resources.DemoVPCTableInternetGatewayRouteDefault.Properties.DestinationCidrBlock: 0.0.0.0/0
734
+ create .Resources.DemoVPCTableInternetGatewayRouteDefault.Properties.GatewayId.Ref: DemoVPCIG
735
+ create .Resources.DemoVPC.Properties.EnableDnsSupport: true
736
+ create .Resources.DemoVPC.Properties.EnableDnsHostnames: true
737
+ ```
738
+
739
+ Everything looks good, so we can converge the stack and create new resources.
740
+
741
+ ```text
742
+ $> convection converge
743
+
744
+ converge Stack vpc
745
+ update_in_progress convection-demo-vpc: (AWS::CloudFormation::Stack) User Initiated
746
+ create_in_progress DemoVPCIG: (AWS::EC2::InternetGateway)
747
+ create_in_progress DemoVPCIG: (AWS::EC2::InternetGateway) Resource creation Initiated
748
+ update_in_progress DemoVPC: (AWS::EC2::VPC)
749
+ update_complete DemoVPC: (AWS::EC2::VPC)
750
+ create_in_progress DemoVPCTableInternetGateway: (AWS::EC2::RouteTable)
751
+ create_in_progress DemoVPCTableInternetGateway: (AWS::EC2::RouteTable) Resource creation Initiated
752
+ create_complete DemoVPCTableInternetGateway: (AWS::EC2::RouteTable)
753
+ create_complete DemoVPCIG: (AWS::EC2::InternetGateway)
754
+ create_in_progress DemoVPCTableInternetGatewayRouteDefault: (AWS::EC2::Route)
755
+ create_in_progress DemoVPCIGVPCAttachmentDemoVPC: (AWS::EC2::VPCGatewayAttachment)
756
+ create_in_progress DemoVPCIGVPCAttachmentDemoVPC: (AWS::EC2::VPCGatewayAttachment) Resource creation Initiated
757
+ create_in_progress DemoVPCTableInternetGatewayRouteDefault: (AWS::EC2::Route) Resource creation Initiated
758
+ create_complete DemoVPCIGVPCAttachmentDemoVPC: (AWS::EC2::VPCGatewayAttachment)
759
+ create_complete DemoVPCTableInternetGatewayRouteDefault: (AWS::EC2::Route)
760
+ update_complete_cleanup_in_progress convection-demo-vpc: (AWS::CloudFormation::Stack)
761
+ update_complete convection-demo-vpc: (AWS::CloudFormation::Stack)
762
+ ```
763
+ Now that we have an internet gateway, we need to associate its route table with
764
+ the public subnet. We can create an
765
+ [AWS::EC2::SubnetRouteTableAssociation][cf-association] resource to do that. The
766
+ `ec2_subnet_route_table_association` method in Convection will do that.
767
+
768
+ Add the below block to the bottom of your vpc.rb template below your `ec2_instance 'NATInstance'` block.
769
+
770
+ ```ruby
771
+ ec2_subnet_route_table_association 'DemoVPCRouteTable' do
772
+ route_table fn_ref('DemoVPCTableInternetGateway')
773
+ subnet fn_ref('PublicSubnet')
774
+ end
775
+ ```
776
+
777
+ Where did the reference to "DemoVPCTableInternetGateway" come from? Convection
778
+ took the name of our VPC "DemoVPC", added "Table" to it, and appended the name
779
+ of our route table "InternetGateway". Looking at the output from our previous
780
+ converge shows the "DemoVPCTableInternetGateway" resource being created. Use
781
+ diff to check that the route table association will be created.
782
+
783
+ ```text
784
+ $> convection diff
785
+
786
+ compare Compare local state of stack vpc (convection-demo-vpc) with remote template
787
+ create .Resources.DemoVPCRouteTable.Type: AWS::EC2::SubnetRouteTableAssociation
788
+ create .Resources.DemoVPCRouteTable.Properties.RouteTableId.Ref: DemoVPCTableInternetGateway
789
+ create .Resources.DemoVPCRouteTable.Properties.SubnetId.Ref: PublicSubnet
790
+ ```
791
+
792
+ Now go ahead and converge the stack.
793
+
794
+ ```text
795
+ $> convection converge
796
+
797
+ converge Stack vpc
798
+ update_in_progress convection-demo-vpc: (AWS::CloudFormation::Stack) User Initiated
799
+ create_in_progress DemoVPCRouteTable: (AWS::EC2::SubnetRouteTableAssociation)
800
+ create_in_progress DemoVPCRouteTable: (AWS::EC2::SubnetRouteTableAssociation) Resource creation Initiated
801
+ create_complete DemoVPCRouteTable: (AWS::EC2::SubnetRouteTableAssociation)
802
+ update_complete_cleanup_in_progress convection-demo-vpc: (AWS::CloudFormation::Stack)
803
+ update_complete convection-demo-vpc: (AWS::CloudFormation::Stack)
804
+ ```
805
+
806
+ ## Create a route table for the private subnet ##
807
+
808
+ Just like we created a route table for the public subnet, we now need a route
809
+ table for the private subnet. Our route table will reference our VPC and define
810
+ a single route for all traffic from our private instances through our NAT.
811
+ Convection's `ec2_route_table` method can be used to explicitly create a route
812
+ table.
813
+
814
+ Add the below block to the bottom of your vpc.rb template
815
+
816
+ ```ruby
817
+ ec2_route_table 'PrivateRouteTable' do
818
+ vpc fn_ref('DemoVPC')
819
+ route 'PrivateRoute' do
820
+ destination '0.0.0.0/0'
821
+ instance fn_ref('NATInstance')
822
+ end
823
+ end
824
+ ```
825
+
826
+ Now we need to link the private route table to the private subnet. Like we did
827
+ for the public subnet, we can use Convection's `ec2_subnet_route_table_association`
828
+ method.
829
+
830
+ Add the below to the bottom of your vpc.rb template.
831
+
832
+ ```ruby
833
+ ec2_subnet_route_table_association 'PrivateRouteAssoc' do
834
+ route_table fn_ref('PrivateRouteTable')
835
+ subnet fn_ref('PrivateSubnet')
836
+ end
837
+
838
+ ```
839
+
840
+ Diffing the template shows three new resources, one for the route, one for the
841
+ route table, and one for the subnet association.
842
+
843
+ ```text
844
+ $> convection diff
845
+
846
+ compare Compare local state of stack vpc (convection-demo-vpc) with remote template
847
+ create .Resources.PrivateRouteTableRoutePrivateRoute.Type: AWS::EC2::Route
848
+ create .Resources.PrivateRouteTableRoutePrivateRoute.Properties.RouteTableId.Ref: PrivateRouteTable
849
+ create .Resources.PrivateRouteTableRoutePrivateRoute.Properties.DestinationCidrBlock: 0.0.0.0/0
850
+ create .Resources.PrivateRouteTableRoutePrivateRoute.Properties.InstanceId.Ref: NATInstance
851
+ create .Resources.PrivateRouteTable.Type: AWS::EC2::RouteTable
852
+ create .Resources.PrivateRouteTable.Properties.VpcId.Ref: DemoVPC
853
+ create .Resources.PrivateRouteAssoc.Type: AWS::EC2::SubnetRouteTableAssociation
854
+ create .Resources.PrivateRouteAssoc.Properties.RouteTableId.Ref: PrivateRouteTable
855
+ create .Resources.PrivateRouteAssoc.Properties.SubnetId.Ref: PrivateSubnet
856
+ ```
857
+
858
+ Converge the stack and create the new resources.
859
+
860
+ ```text
861
+ $> convection converge
862
+
863
+ converge Stack vpc
864
+ update_in_progress convection-demo-vpc: (AWS::CloudFormation::Stack) User Initiated
865
+ create_in_progress PrivateRouteTable: (AWS::EC2::RouteTable)
866
+ create_in_progress PrivateRouteTable: (AWS::EC2::RouteTable) Resource creation Initiated
867
+ create_complete PrivateRouteTable: (AWS::EC2::RouteTable)
868
+ create_in_progress PrivateRouteTableRoutePrivateRoute: (AWS::EC2::Route)
869
+ create_in_progress PrivateRouteAssoc: (AWS::EC2::SubnetRouteTableAssociation)
870
+ create_in_progress PrivateRouteTableRoutePrivateRoute: (AWS::EC2::Route) Resource creation Initiated
871
+ create_in_progress PrivateRouteAssoc: (AWS::EC2::SubnetRouteTableAssociation) Resource creation Initiated
872
+ create_complete PrivateRouteAssoc: (AWS::EC2::SubnetRouteTableAssociation)
873
+ create_complete PrivateRouteTableRoutePrivateRoute: (AWS::EC2::Route)
874
+ update_complete_cleanup_in_progress convection-demo-vpc: (AWS::CloudFormation::Stack)
875
+ update_complete convection-demo-vpc: (AWS::CloudFormation::Stack)
876
+ ```
877
+
878
+ ## Where to go from here ##
879
+
880
+ We used Convection to build an Amazon VPC with public and private subnets,
881
+ complete with a NAT router for handling internet traffic and a security group
882
+ for locking down access. From here we could add bastion servers to get SSH
883
+ access to EC2 instances in the private subnet, or network ACLs to further harden
884
+ the VPC.
885
+
886
+ Whatever we do, we now have a solid workflow for making infrastructure
887
+ improvements. Make a small change. Diff to see what's going to be updated.
888
+ Converge the change if it looks good.
889
+
890
+
891
+ [vpc2]: http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Scenario2.html
892
+ [cf-template]: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-reference.html
893
+ [cf-vpc]: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-vpc.html
894
+ [cf-subnet]: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-subnet.html
895
+ [ec2-subnet]: https://github.com/rapid7/convection/blob/v0.2/lib/convection/model/template/resource/aws_ec2_subnet.rb
896
+ [cf-ref]: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-ref.html
897
+ [cf-ec2]: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-instance.html
898
+ [cf-security-group]: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-security-group.html
899
+ [cf-internet-gateway]: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-internet-gateway.html
900
+ [cf-route-table]: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-route-table.html
901
+ [cf-gateway-attachment]: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-vpc-gateway-attachment.html
902
+ [cf-route]: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-route.html
903
+ [cf-association]: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-subnet-route-table-assoc.html
904
+ [nat-instance]: http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_NAT_Instance.html