sparkle_formation 0.2.2 → 0.2.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (42) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +6 -0
  3. data/README.md +7 -1
  4. data/bin/generate_sparkle_docs +31 -0
  5. data/lib/sparkle_formation/aws.rb +6 -7
  6. data/lib/sparkle_formation/sparkle_attribute.rb +97 -0
  7. data/lib/sparkle_formation/translation/heat.rb +184 -2
  8. data/lib/sparkle_formation/translation/rackspace.rb +7 -3
  9. data/lib/sparkle_formation/translation.rb +28 -2
  10. data/lib/sparkle_formation/version.rb +1 -1
  11. data/sparkle_formation.gemspec +2 -1
  12. metadata +5 -33
  13. data/CONTRIBUTING.md +0 -25
  14. data/Gemfile +0 -3
  15. data/Gemfile.lock +0 -18
  16. data/bin/aws_resources +0 -85
  17. data/bin/heat_resources +0 -33
  18. data/docs/README.md +0 -177
  19. data/docs/anatomy.md +0 -203
  20. data/docs/building-blocks.md +0 -275
  21. data/docs/examples/cloudformation/components/base.rb +0 -25
  22. data/docs/examples/cloudformation/dynamics/elb.rb +0 -23
  23. data/docs/examples/cloudformation/templates/website.rb +0 -30
  24. data/docs/examples/template_json/website.json +0 -88
  25. data/docs/examples/website.rb +0 -74
  26. data/docs/functions.md +0 -41
  27. data/docs/properties.md +0 -32
  28. data/docs/provisioning.md +0 -82
  29. data/docs/resource-reference.md +0 -49
  30. data/examples/allinone/cloudformation/ec2_example.rb +0 -59
  31. data/examples/allinone/parse.rb +0 -14
  32. data/examples/ami_component/cloudformation/components/ami.rb +0 -21
  33. data/examples/ami_component/cloudformation/ec2_example.rb +0 -39
  34. data/examples/ami_component/parse.rb +0 -14
  35. data/test/spec.rb +0 -6
  36. data/test/specs/attribute.rb +0 -72
  37. data/test/specs/basic.rb +0 -76
  38. data/test/specs/cloudformation/components/ami.rb +0 -14
  39. data/test/specs/cloudformation/dynamics/node.rb +0 -16
  40. data/test/specs/results/base.json +0 -43
  41. data/test/specs/results/base_with_map.json +0 -50
  42. data/test/specs/results/component.json +0 -27
@@ -1,275 +0,0 @@
1
- ## SparkleFormation Building Blocks
2
-
3
- Using SparkleFormation for the above template has already saved us
4
- many keystrokes, but what about reusing SparkleFormation code between
5
- similar stacks? This is where SparkleFormation concepts like
6
- components, dynamics, and registries come into play.
7
-
8
- ### Components
9
-
10
- Components are static configuration which can be reused between many
11
- stack templates. In our example case we have decided that all our
12
- stacks will need to make use of IAM credentials, so we will create
13
- a component which allows us to inserts the two IAM resources into any
14
- template in a resuable fashion. The component, which we will call
15
- 'base' and put in a file called 'base.rb,' would look like this:
16
-
17
- ```ruby
18
- SparkleFormation.build do
19
- set!('AWSTemplateFormatVersion', '2010-09-09')
20
-
21
- resources.cfn_user do
22
- type 'AWS::IAM::User'
23
- properties.path '/'
24
- properties.policies _array(
25
- -> {
26
- policy_name 'cfn_access'
27
- policy_document.statement _array(
28
- -> {
29
- effect 'Allow'
30
- action 'cloudformation:DescribeStackResource'
31
- resource '*'
32
- }
33
- )
34
- }
35
- )
36
- end
37
-
38
- resources.cfn_keys do
39
- type 'AWS::IAM::AccessKey'
40
- properties.user_name ref!(:cfn_user)
41
- end
42
- end
43
- ```
44
-
45
- After moving these resources out of the initial template and into a
46
- component, we will update the template so that the base component is
47
- loaded on the first line, and the resources it contains are no longer
48
- present in the template itself:
49
-
50
- ```ruby
51
- SparkleFormation.new(:website).load(:base).overrides do
52
-
53
- description 'Supercool Website'
54
-
55
- parameters.web_nodes do
56
- type 'Number'
57
- description 'Number of web nodes for ASG.'
58
- default '2'
59
- end
60
-
61
- resources.website_autoscale do
62
- type 'AWS::AutoScaling::AutoScalingGroup'
63
- properties do
64
- availability_zones({'Fn::GetAZs' => ''})
65
- launch_configuration_name ref!(:website_launch_config)
66
- min_size ref!(:web_nodes)
67
- max_size ref!(:web_nodes)
68
- end
69
- end
70
-
71
- resources.website_launch_config do
72
- type 'AWS::AutoScaling::LaunchConfiguration'
73
- properties do
74
- image_id 'ami-123456'
75
- instance_type 'm3.medium'
76
- end
77
- end
78
-
79
- resources.website_elb do
80
- type 'AWS::ElasticLoadBalancing::LoadBalancer'
81
- properties do
82
- availability_zones._set('Fn::GetAZs', '')
83
- listeners _array(
84
- -> {
85
- load_balancer_port '80'
86
- protocol 'HTTP'
87
- instance_port '80'
88
- instance_protocol 'HTTP'
89
- }
90
- )
91
- health_check do
92
- target 'HTTP:80/'
93
- healthy_threshold '3'
94
- unhealthy_threshold '3'
95
- interval '5'
96
- timeout '15'
97
- end
98
- end
99
- end
100
- end
101
- ```
102
-
103
- ### Dynamics
104
-
105
- Like components, dynamics are another SparkleFormation feature which
106
- enables code reuse between stack templates. Where components are
107
- static, dynamics are useful for creating unique resources via
108
- the passing of arguments.
109
-
110
- In our example scenario we have decided that we want to use elastic
111
- load balancer resources in many of our stack templates, we want to
112
- create a dynamic which makes inserting ELB resources much easier than
113
- copying the full resource configuration between templates.
114
-
115
- The resulting implementation would look something like this:
116
-
117
- ```ruby
118
- SparkleFormation.dynamic(:elb) do |_name, _config={}|
119
- resources("#{_name}_elb".to_sym) do
120
- type 'AWS::ElasticLoadBalancing::LoadBalancer'
121
- properties do
122
- availability_zones._set('Fn::GetAZs', '')
123
- listeners _array(
124
- -> {
125
- load_balancer_port _config[:load_balancer_port] || '80'
126
- protocol _config[:protocol] || 'HTTP'
127
- instance_port _config[:instance_port] || '80'
128
- instance_protocol _config[:instance_protocol] || 'HTTP'
129
- }
130
- )
131
- health_check do
132
- target _config[:target] || 'HTTP:80/'
133
- healthy_threshold _config[:healthy_threshold] || '3'
134
- unhealthy_threshold _config[:unhealthy_threshold] || '3'
135
- interval _config[:interval] || '5'
136
- timeout _config[:timeout] || '15'
137
- end
138
- end
139
- end
140
- end
141
- ```
142
-
143
- This dynamic accepts two arguments: `_name` (a string, required) and `_config`
144
- (a hash, optional). The dynamic will use the values passed in these
145
- arguments to generate a new ELB resource, and override the default ELB
146
- properties wherever a corresponding key/value pair is provided in the
147
- `_config` hash.
148
-
149
- Once updated to make use of the new ELB dynamic, our template looks
150
- like this:
151
-
152
- ```ruby
153
- SparkleFormation.new(:website).load(:base).overrides do
154
-
155
- description 'Supercool Website'
156
-
157
- parameters.web_nodes do
158
- type 'Number'
159
- description 'Number of web nodes for ASG.'
160
- default '2'
161
- end
162
-
163
- resources.website_autoscale do
164
- type 'AWS::AutoScaling::AutoScalingGroup'
165
- properties do
166
- availability_zones({'Fn::GetAZs' => ''})
167
- launch_configuration_name ref!(:website_launch_config)
168
- min_size ref!(:web_nodes)
169
- max_size ref!(:web_nodes)
170
- end
171
- end
172
-
173
- resources.website_launch_config do
174
- type 'AWS::AutoScaling::LaunchConfiguration'
175
- properties do
176
- image_id 'ami-123456'
177
- instance_type 'm3.medium'
178
- end
179
- end
180
-
181
- dynamic!(:elb, 'website')
182
- end
183
- ```
184
-
185
- If we wanted to override the default configuration for the ELB,
186
- e.g. to configure the ELB to listen on and communicate with back-end
187
- node on port 8080 instead of 80, we can specify these override values
188
- in the configuration passed to the ELB dynamic:
189
-
190
- ```ruby
191
- dynamic!(:elb, 'website', :load_balancer_port => 8080, :instance_port => 8080)
192
- ```
193
-
194
- We're passing three arguments here:
195
-
196
- 1. `:elb` is the name of the dynamic to insert, as a ruby symbol
197
- 2. A name string (`'website'`), which is passed to `_name` in the
198
- dynamic. This is prepended to the resource name in the dynamic,
199
- resulting in a unique resource name.
200
- 3. The ELB ports to configure as key/value pairs. These are passed into the dynamic as
201
- the `_config` hash.
202
-
203
- ### Registries
204
-
205
- Similar to dynamics, registries are reusable resource configuration
206
- code which can be reused inside different resource definitions.
207
-
208
- Registries are useful for defining properties that may be reused
209
- between resources of different types. For example, the
210
- LaunchConfiguration and Instance resource types make use of metadata
211
- properties which inform both resource types how to bootstrap one or
212
- more instances.
213
-
214
- In our example scenario we would like our new instances to run
215
- 'sudo apt-get update && sudo apt-get upgrade -y' at first boot,
216
- regardless of whether or not the instances are members of an
217
- autoscaling group.
218
-
219
- Because these resources are of different types, placing the metadata
220
- required for this task directly inside a dynamic isn't going to work
221
- quite the way we need. Instead we can put this inside a registry which
222
- can be inserted into the resources defined in one or more dynamics:
223
-
224
- ```ruby
225
- SparkleFormation::Registry.register(:apt_get_update) do
226
- metadata('AWS::CloudFormation::Init') do
227
- _camel_keys_set(:auto_disable) do
228
- commands('01_apt_get_update') do
229
- command 'sudo apt-get update'
230
- end
231
- commands('02_apt_get_upgrade') do
232
- command 'sudo apt-get upgrade -y'
233
- end
234
- end
235
- end
236
- ```
237
-
238
- Now we can insert this registry entry into our existing template, to
239
- ensure that apt is updated upon provisioning:
240
-
241
- ```ruby
242
- SparkleFormation.new(:website).load(:base).overrides do
243
-
244
- description 'Supercool Website'
245
-
246
- parameters.web_nodes do
247
- type 'Number'
248
- description 'Number of web nodes for ASG.'
249
- default 2
250
- end
251
-
252
- resources.website_autoscale do
253
- type 'AWS::AutoScaling::AutoScalingGroup'
254
- properties do
255
- availability_zones({'Fn::GetAZs' => ''})
256
- launch_configuration_name ref!(:website_launch_config)
257
- min_size ref!(:web_nodes)
258
- max_size ref!(:web_nodes)
259
- end
260
- end
261
-
262
- resources.website_launch_config do
263
- type 'AWS::AutoScaling::LaunchConfiguration'
264
- registry!(:apt_get_update, 'website')
265
- properties do
266
- image_id 'ami-123456'
267
- instance_type 'm3.medium'
268
- end
269
- end
270
-
271
- dynamic!(:elb, 'website')
272
-
273
- registry!(:apt_get_update, 'website')
274
- end
275
- ```
@@ -1,25 +0,0 @@
1
- SparkleFormation.build do
2
- set!('AWSTemplateFormatVersion', '2010-09-09')
3
-
4
- resources.cfn_user do
5
- type 'AWS::IAM::User'
6
- properties.path '/'
7
- properties.policies _array(
8
- -> {
9
- policy_name 'cfn_access'
10
- policy_document.statement _array(
11
- -> {
12
- effect 'Allow'
13
- action 'cloudformation:DescribeStackResource'
14
- resource '*'
15
- }
16
- )
17
- }
18
- )
19
- end
20
-
21
- resources.cfn_keys do
22
- type 'AWS::IAM::AccessKey'
23
- properties.user_name ref!(:cfn_user)
24
- end
25
- end
@@ -1,23 +0,0 @@
1
- SparkleFormation.dynamic(:elb) do |_name, _config={}|
2
- resources("#{_name}_elb".to_sym) do
3
- type 'AWS::ElasticLoadBalancing::LoadBalancer'
4
- properties do
5
- availability_zones._set('Fn::GetAZs', '')
6
- listeners _array(
7
- -> {
8
- load_balancer_port _config[:load_balancer_port] || '80'
9
- protocol _config[:protocol] || 'HTTP'
10
- instance_port _config[:instance_port] || '80'
11
- instance_protocol _config[:instance_protocol] || 'HTTP'
12
- }
13
- )
14
- health_check do
15
- target _config[:target] || 'HTTP:80/'
16
- healthy_threshold _config[:healthy_threshold] || '3'
17
- unhealthy_threshold _config[:unhealthy_threshold] || '3'
18
- interval _config[:interval] || '5'
19
- timeout _config[:timeout] || '15'
20
- end
21
- end
22
- end
23
- end
@@ -1,30 +0,0 @@
1
- SparkleFormation.new(:website).load(:base).overrides do
2
-
3
- description 'Supercool Website'
4
-
5
- parameters.web_nodes do
6
- type 'Number'
7
- description 'Number of web nodes for ASG.'
8
- default 2
9
- end
10
-
11
- resources.website_autoscale do
12
- type 'AWS::AutoScaling::AutoScalingGroup'
13
- properties do
14
- availability_zones({'Fn::GetAZs' => ''})
15
- launch_configuration_name ref!(:website_launch_config)
16
- min_size ref!(:web_nodes)
17
- max_size ref!(:web_nodes)
18
- end
19
- end
20
-
21
- resources.website_launch_config do
22
- type 'AWS::AutoScaling::LaunchConfiguration'
23
- properties do
24
- image_id 'ami-123456'
25
- instance_type 'm3.medium'
26
- end
27
- end
28
-
29
- dynamic!(:elb, 'website')
30
- end
@@ -1,88 +0,0 @@
1
- {
2
- "AWSTemplateFormatVersion": "2010-09-09",
3
- "Description": "Supercool Website",
4
- "Resources": {
5
- "CfnUser": {
6
- "Type": "AWS::IAM::User",
7
- "Properties": {
8
- "Path": "/",
9
- "Policies": [
10
- {
11
- "PolicyName": "cfn_access",
12
- "PolicyDocument": {
13
- "Statement": [
14
- {
15
- "Effect": "Allow",
16
- "Action": "cloudformation:DescribeStackResource",
17
- "Resource": "*"
18
- }
19
- ]
20
- }
21
- }
22
- ]
23
- }
24
- },
25
- "CfnKeys": {
26
- "Type": "AWS::IAM::AccessKey",
27
- "Properties": {
28
- "UserName": {
29
- "Ref": "CfnUser"
30
- }
31
- }
32
- },
33
- "WebsiteAutoscale": {
34
- "Type": "AWS::AutoScaling::AutoScalingGroup",
35
- "Properties": {
36
- "AvailabilityZones": {
37
- "Fn::GetAZs": ""
38
- },
39
- "LaunchConfigurationName": {
40
- "Ref": "WebsiteLaunchConfig"
41
- },
42
- "MinSize": {
43
- "Ref": "WebNodes"
44
- },
45
- "MaxSize": {
46
- "Ref": "WebNodes"
47
- }
48
- }
49
- },
50
- "WebsiteLaunchConfig": {
51
- "Type": "AWS::AutoScaling::LaunchConfiguration",
52
- "Properties": {
53
- "ImageId": "ami-123456",
54
- "InstanceType": "m3.medium"
55
- }
56
- },
57
- "WebsiteElb": {
58
- "Type": "AWS::ElasticLoadBalancing::LoadBalancer",
59
- "Properties": {
60
- "AvailabilityZones": {
61
- "Fn::GetAZs": ""
62
- },
63
- "Listeners": [
64
- {
65
- "LoadBalancerPort": "80",
66
- "Protocol": "HTTP",
67
- "InstancePort": "80",
68
- "InstanceProtocol": "HTTP"
69
- }
70
- ],
71
- "HealthCheck": {
72
- "Target": "HTTP:80/",
73
- "HealthyThreshold": "3",
74
- "UnhealthyThreshold": "3",
75
- "Interval": "5",
76
- "Timeout": "15"
77
- }
78
- }
79
- }
80
- },
81
- "Parameters": {
82
- "WebNodes": {
83
- "Type": "Number",
84
- "Description": "Number of web nodes for ASG.",
85
- "Default": "2"
86
- }
87
- }
88
- }
@@ -1,74 +0,0 @@
1
- SparkleFormation.new('website') do
2
-
3
- set!('AWSTemplateFormatVersion', '2010-09-09')
4
-
5
- description 'Supercool Website'
6
-
7
- resources.cfn_user do
8
- type 'AWS::IAM::User'
9
- properties.path '/'
10
- properties.policies _array(
11
- -> {
12
- policy_name 'cfn_access'
13
- policy_document.statement _array(
14
- -> {
15
- effect 'Allow'
16
- action 'cloudformation:DescribeStackResource'
17
- resource '*'
18
- }
19
- )
20
- }
21
- )
22
- end
23
-
24
- resources.cfn_keys do
25
- type 'AWS::IAM::AccessKey'
26
- properties.user_name ref!(:cfn_user)
27
- end
28
-
29
- parameters.web_nodes do
30
- type 'Number'
31
- description 'Number of web nodes for ASG.'
32
- default 2
33
- end
34
-
35
- resources.website_autoscale do
36
- type 'AWS::AutoScaling::AutoScalingGroup'
37
- properties do
38
- availability_zones({'Fn::GetAZs' => ''})
39
- launch_configuration_name ref!(:website_launch_config)
40
- min_size ref!(:web_nodes)
41
- max_size ref!(:web_nodes)
42
- end
43
- end
44
-
45
- resources.website_launch_config do
46
- type 'AWS::AutoScaling::LaunchConfiguration'
47
- properties do
48
- image_id 'ami-123456'
49
- instance_type 'm3.medium'
50
- end
51
- end
52
-
53
- resources.website_elb do
54
- type 'AWS::ElasticLoadBalancing::LoadBalancer'
55
- properties do
56
- availability_zones._set('Fn::GetAZs', '')
57
- listeners _array(
58
- -> {
59
- load_balancer_port '80'
60
- protocol 'HTTP'
61
- instance_port '80'
62
- instance_protocol 'HTTP'
63
- }
64
- )
65
- health_check do
66
- target 'HTTP:80/'
67
- healthy_threshold '3'
68
- unhealthy_threshold '3'
69
- interval '5'
70
- timeout '15'
71
- end
72
- end
73
- end
74
- end
data/docs/functions.md DELETED
@@ -1,41 +0,0 @@
1
- ## Intrinsic Functions
2
- The following are all intrinsic AWS Cloudformation functions that are
3
- supported with special syntax in SparkleFormation. Note that these may
4
- not be implemented for all providers.
5
-
6
- ### Ref
7
- Ref allows you to reference parameter and resource values. We did this
8
- earlier with the autoscaling group size:
9
- ```ruby
10
- parameters.web_nodes do
11
- type 'Number'
12
- description 'Number of web nodes for ASG.'
13
- default '2'
14
- end
15
-
16
- ...
17
-
18
- min_size ref!(:web_nodes)
19
- ```
20
- It also works for resource names. The following returns the name of
21
- the launch configuration so we can use it in the autoscaling group
22
- properties.
23
- ```ruby
24
- ref!(:website_launch_config)
25
- ```
26
-
27
- ### Join
28
- A Join combines strings. You can use Refs and Mappings within a Join.
29
- ```ruby
30
- join!(ref!(:environment), '-', map!(:region_map, ref!('AWS::Region'), :ami))
31
- ```
32
- Would return 'development-us-east-1', if we built a stack in the
33
- AWS Virgnia region and provided 'development' for the environment
34
- parameter.
35
-
36
- ### Attr
37
- Certain resources attributes can be retrieved directly. To access an
38
- IAM user's (in this case, :cfn_user) secret key:
39
- ```ruby
40
- attr!(:cfn_user, :secret_access_key)
41
- ```
data/docs/properties.md DELETED
@@ -1,32 +0,0 @@
1
- ## Universal Properties
2
-
3
- ### Tags
4
- Tags can be applied to most resources. These make it easy to track
5
- resource usage across stacks. They may be used for cost tracking as
6
- well as configuration tools that are cloud-infrastructure aware. Tags
7
- are provided as key/value pairs within an array. In this example we
8
- provide the stack name and a contact email:
9
-
10
- ```ruby
11
- resources.website_autoscale do
12
- type 'AWS::AutoScaling::AutoScalingGroup'
13
- properties do
14
- availability_zones({ 'Fn::GetAZs' => '' })
15
- tags _array(
16
- -> {
17
- key 'StackName'
18
- value ref!('AWS::StackName'))
19
- propagate_at_launch true
20
- },
21
- -> {
22
- key 'ContactEmail'
23
- value 'support@hw-ops.com'
24
- propagate_at_launch true
25
- }
26
- )
27
- launch_configuration_name ref!(:website_launch_config)
28
- end
29
- ```
30
-
31
- Please check the relevant reference documentation to confirm that tags
32
- are available for a specific resource type.
data/docs/provisioning.md DELETED
@@ -1,82 +0,0 @@
1
- ## Provisioning SparkleFormations
2
-
3
- ### JSON Templates
4
-
5
- Generating JSON from a SparkleFormation template is as simple as
6
- calling `SparkleFormation.compile()` on the template file. Here's a simple
7
- script to output a JSON template from a supplied
8
- SparkeFormation template:
9
-
10
- ```ruby
11
- #!/usr/bin/env ruby
12
- require 'sparkle_formation'
13
- require 'json'
14
-
15
- puts SparkleFormation.compile(ARGV[0])
16
- ```
17
-
18
- The output can be written to a file and uploaded to the provider using
19
- the method of your choice.
20
-
21
- For a more legible template:
22
-
23
- ```ruby
24
- puts JSON.pretty_generate(SparkleFormation.compile(ARGV[0]))
25
- ```
26
-
27
- Note: The output from this command may not be usable with cloud providers,
28
- as the many spaces and newlines may exceed the cloudformation
29
- character limit. However, it is much easier to read.
30
-
31
- ### Knife Cloudformation
32
- knife-cloudformation [knife-cloudformation plugin](https://rubygems.org/gems/knife-cloudformation) is a plugin for
33
- knife that provisions cloudformation stacks. It can be used with
34
- SparkleFormation to build stacks without the intermediary steps of
35
- writing a JSON template to file and uploading the template to the provider.
36
-
37
- #### Processing SparkleFormation Templates
38
- To build a stack directly from a SparkleFormation template, use the
39
- `create` command with the `--file` and `--processing` flags:
40
-
41
- ```
42
- knife cloudformation create my-web-stack --file templates/website.rb --processing
43
- ```
44
-
45
- `--file` directs knife to a file under the `cloudformation` directory,
46
- and `--processing` tells knife to render JSON from the
47
- SparkleFormation template before passing it to the provider.
48
-
49
- #### Applying Stacks
50
- You can also apply an existing stack's outputs to the stack you are
51
- building. Using the `--apply-stack` flag sets parameters to the
52
- values of any matching outputs.
53
-
54
- Consider that you have built a database stack (`db-stack-01`) that includes an output for the
55
- database endpoint:
56
-
57
- ```ruby
58
- outputs do
59
- database_endpoint do
60
- value attr!(:database_elb, 'DNSName')
61
- description "Database ELB Endpoint for client connections"
62
- end
63
- end
64
- ```
65
-
66
- Next, you build a website stack (`web-stack-01`) that needs to connect to the
67
- database. The SparkleFormation for this stack includes a parameter to
68
- prompt for the database endpoint:
69
-
70
- ```ruby
71
- parameters.database_endpoint do
72
- type 'String'
73
- description 'Database endpoint to connect to'
74
- default 'localhost'
75
- end
76
- ```
77
-
78
- Using knife-cloudformation, you apply the database stack in order to
79
- automatically provide the correct database endpoint:
80
-
81
- `knife cloudformation create web-stack-01 --file templates/website.rb --processing --apply-stack db-stack-01`
82
-