sparkle-pack-jackal-cfn 0.1.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- data/CHANGELOG.md +2 -0
- data/LICENSE +13 -0
- data/README.md +129 -0
- data/lib/sparkle-pack-jackal-cfn.rb +3 -0
- data/lib/sparkleformation/dynamics/ami_manager.rb +8 -0
- data/lib/sparkleformation/dynamics/ami_register.rb +8 -0
- data/lib/sparkleformation/dynamics/hash_extractor.rb +8 -0
- data/lib/sparkleformation/dynamics/jackal_image.rb +84 -0
- data/lib/sparkleformation/dynamics/jackal_stack.rb +8 -0
- data/lib/sparkleformation/dynamics/orchestration_unit.rb +8 -0
- data/lib/sparkleformation/jackal_bus.rb +46 -0
- data/lib/sparkleformation/jackal_cfn.rb +371 -0
- data/lib/sparkleformation/jackal_usage_example.rb +15 -0
- data/sparkle-pack-jackal-cfn.gemspec +13 -0
- metadata +72 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: 68eac1e8d59b0d0c7ce62f3c8e467313e8c50bf6
|
4
|
+
data.tar.gz: 73e0fb3a5760fd6a5eaf1ff1a82f3c3f6d9d9f5a
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: 56b6ba743aff42df8bf5ff15b67fea0e8cbecf162f93de577c7cfd58d13406cd8fac42fa0b7140b7de53416e6ad695827f2a29200cdfa35455fe29c9c09c8760
|
7
|
+
data.tar.gz: 7e685a95dcf3b686bbf41e455d5f5962d05a755bea52e3e0f9fc1cd9b5efadfcac9301cd328e4700188e61e495c793d2d079520cc77811fc96262ea5f3ead498
|
data/CHANGELOG.md
ADDED
data/LICENSE
ADDED
@@ -0,0 +1,13 @@
|
|
1
|
+
Copyright 2016 Chris Roberts
|
2
|
+
|
3
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
4
|
+
you may not use this file except in compliance with the License.
|
5
|
+
You may obtain a copy of the License at
|
6
|
+
|
7
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
8
|
+
|
9
|
+
Unless required by applicable law or agreed to in writing, software
|
10
|
+
distributed under the License is distributed on an "AS IS" BASIS,
|
11
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
12
|
+
See the License for the specific language governing permissions and
|
13
|
+
limitations under the License.
|
data/README.md
ADDED
@@ -0,0 +1,129 @@
|
|
1
|
+
# SparklePack - Jackal CFN
|
2
|
+
|
3
|
+
This is a [SparkleFormation][1] [SparklePack][2] for [jackal-cfn][3]. It provides
|
4
|
+
integrations for the custom resources provided by the [jackal-cfn][3]
|
5
|
+
library as well as templates for worker setup and example usage.
|
6
|
+
|
7
|
+
## Integrations
|
8
|
+
|
9
|
+
### Dynamics
|
10
|
+
|
11
|
+
This [SparklePack][2] provides dynamics for custom resources provided
|
12
|
+
by [jackal-cfn][3]:
|
13
|
+
|
14
|
+
* `:ami_manager`
|
15
|
+
* `:ami_register`
|
16
|
+
* `:hash_extractor`
|
17
|
+
* `:jackal_stack`
|
18
|
+
* `:orchestration_unit`
|
19
|
+
|
20
|
+
It also provides a customized `:jackal_image` dynamic which will
|
21
|
+
create the resource as well as an output of the newly created
|
22
|
+
AMI ID.
|
23
|
+
|
24
|
+
### Templates
|
25
|
+
|
26
|
+
This [SparklePack][2] includes 3 templates:
|
27
|
+
|
28
|
+
1. `jackal_cfn` - Creates a working processor to handle requests
|
29
|
+
2. `jackal_bus` - Creates _only_ the required message bus resources
|
30
|
+
3. `jackal_usage_example` - Example template to display usage of the OrchestrationUnit resource
|
31
|
+
|
32
|
+
_NOTE: The `jackal_cfn` template only supports the follow regions:_
|
33
|
+
|
34
|
+
* ap-northeast-1
|
35
|
+
* ap-southeast-1
|
36
|
+
* ap-southeast-2
|
37
|
+
* cn-north-1
|
38
|
+
* eu-central-1
|
39
|
+
* eu-west-1
|
40
|
+
* sa-east-1
|
41
|
+
* us-east-1
|
42
|
+
* us-west-1
|
43
|
+
* us-west-2
|
44
|
+
* us-gov-west-1
|
45
|
+
|
46
|
+
## Usage
|
47
|
+
|
48
|
+
_NOTE: Assumes a working `sfn` setup. (see: [Getting started guide][4])_
|
49
|
+
|
50
|
+
### Setup
|
51
|
+
|
52
|
+
First, include the [SparklePack][2] into the bundle by adding this line to
|
53
|
+
the `./Gemfile`:
|
54
|
+
|
55
|
+
```ruby
|
56
|
+
gem 'sparkle-pack-jackal-cfn'
|
57
|
+
```
|
58
|
+
|
59
|
+
Next, update the local bundle:
|
60
|
+
|
61
|
+
```
|
62
|
+
$ bundle update
|
63
|
+
```
|
64
|
+
|
65
|
+
Now enable the [SparklePack][2] within the `.sfn` configuration file:
|
66
|
+
|
67
|
+
```ruby
|
68
|
+
Configuration.new do
|
69
|
+
...
|
70
|
+
sparkle_pack 'sparkle-pack-jackal-cfn'
|
71
|
+
...
|
72
|
+
end
|
73
|
+
```
|
74
|
+
|
75
|
+
### Build a jackal-cfn processor
|
76
|
+
|
77
|
+
To use the [jackal-cfn][3] resources, a processor instance must be available. The
|
78
|
+
[SparklePack][2] provides a template to build a processor stack. This stack will
|
79
|
+
build a single EC2 instance which will be used as the base for an on demand
|
80
|
+
autoscaling group. To create the stack:
|
81
|
+
|
82
|
+
```
|
83
|
+
$ bundle exec sfn create STACK_NAME_PROCESSOR --file jackal_cfn
|
84
|
+
```
|
85
|
+
|
86
|
+
### Using the jackal-cfn processor
|
87
|
+
|
88
|
+
This [SparklePack][2] includes an example template that uses the `OrchestrationUnit`
|
89
|
+
resource to read the contents of a file on the running processor instance. The
|
90
|
+
contents of the file is a JSON serialized Hash that includes the ID of the
|
91
|
+
current stack. The example stack will return the full result as well as the
|
92
|
+
specific `"OriginStack"` value in the outputs:
|
93
|
+
|
94
|
+
```
|
95
|
+
$ bundle exec sfn create STACK_NAME_EXAMPLE --file jackal_usage_example --apply-stack STACK_NAME_PROCESSOR
|
96
|
+
```
|
97
|
+
|
98
|
+
### All in one
|
99
|
+
|
100
|
+
The templates in this [SparklePack][2] support nesting. To create a single stack
|
101
|
+
that includes the `jackal_cfn` template and the `jackal_usage_example` template,
|
102
|
+
create a new local template:
|
103
|
+
|
104
|
+
```ruby
|
105
|
+
# ./sparkleformation/full_jackal.rb
|
106
|
+
|
107
|
+
SparkleFormation.new(:full_jackal) do
|
108
|
+
nest!(:jackal_cfn)
|
109
|
+
nest!(:jackal_usage_example)
|
110
|
+
end
|
111
|
+
```
|
112
|
+
|
113
|
+
And create the nested stacks:
|
114
|
+
|
115
|
+
```
|
116
|
+
$ bundle exec sfn create STACK_NAME_ALL --file full_stack
|
117
|
+
```
|
118
|
+
|
119
|
+
## Info
|
120
|
+
|
121
|
+
* Repository: https://github.com/carnivore-rb/sparkle-pack-jackal-cfn
|
122
|
+
* Jackal CFN: https://github.com/carnivore-rb/jackal-cfn
|
123
|
+
* SparkleFormation: http://www.sparkleformation.io
|
124
|
+
* IRC: Freenode @ #carnivore
|
125
|
+
|
126
|
+
[1]: http://www.sparkleformation.io/
|
127
|
+
[2]: http://www.sparkleformation.io/docs/sparkle_formation/sparkle-packs.html
|
128
|
+
[3]: https://github.com/carnivore-rb/jackal-cfn
|
129
|
+
[4]: http://www.sparkleformation.io/docs/guides/getting-started.html
|
@@ -0,0 +1,84 @@
|
|
1
|
+
SparkleFormation.dynamic(:jackal_image) do |name, opts={}|
|
2
|
+
|
3
|
+
ec2_required_info = [:image_id, :key_name, :instance_type]
|
4
|
+
|
5
|
+
ec2_required_info.each do |req_value|
|
6
|
+
unless(opts[req_value])
|
7
|
+
parameters.set!("#{name}_#{req_value}".to_sym).type 'String'
|
8
|
+
end
|
9
|
+
end
|
10
|
+
|
11
|
+
unless(opts[:service_token])
|
12
|
+
parameters.jackal_service_token.type 'String'
|
13
|
+
end
|
14
|
+
|
15
|
+
base_instance = dynamic!(:ec2_instance, "#{name}_jackal_image".to_sym) do
|
16
|
+
properties do
|
17
|
+
ec2_required_info.each do |req_value|
|
18
|
+
set!(req_value, opts.fetch(req_value, ref!("#{name}_#{req_value}".to_sym)))
|
19
|
+
end
|
20
|
+
if(opts[:user_data])
|
21
|
+
user_data opts[:user_data]
|
22
|
+
else
|
23
|
+
user_data base64!(
|
24
|
+
join!(
|
25
|
+
"#!/bin/bash\n",
|
26
|
+
"apt-get update\n",
|
27
|
+
"apt-get -y install python-setuptools\n",
|
28
|
+
"easy_install https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz\n",
|
29
|
+
'/usr/local/bin/cfn-init -v --region ',
|
30
|
+
region!,
|
31
|
+
' -s ',
|
32
|
+
stack_name!,
|
33
|
+
" -r #{resource_name!}\n",
|
34
|
+
"cfn-signal -e $? -r 'Provision complete' --resource #{resource_name!} --region ",
|
35
|
+
region!,
|
36
|
+
' --stack ',
|
37
|
+
stack_name!
|
38
|
+
)
|
39
|
+
)
|
40
|
+
end
|
41
|
+
end
|
42
|
+
unless(opts[:user_data])
|
43
|
+
creation_policy.resource_signal do
|
44
|
+
count 1
|
45
|
+
timeout 'PT30M'
|
46
|
+
end
|
47
|
+
end
|
48
|
+
end
|
49
|
+
|
50
|
+
resources.set!("#{name}_jackal_image".to_sym) do
|
51
|
+
type 'Custom::AmiRegister'
|
52
|
+
properties do
|
53
|
+
service_token opts.fetch(:service_token, ref!(:jackal_service_token))
|
54
|
+
parameters do
|
55
|
+
name join!(
|
56
|
+
"jackal-image-#{name}",
|
57
|
+
opts.fetch(
|
58
|
+
:image_instance_id,
|
59
|
+
ref!("#{name}_jackal_image_ec2_instance".to_sym)
|
60
|
+
),
|
61
|
+
stack_name!,
|
62
|
+
:options => {
|
63
|
+
:delimiter => '-'
|
64
|
+
}
|
65
|
+
)
|
66
|
+
instance_id opts.fetch(
|
67
|
+
:image_instance_id,
|
68
|
+
ref!("#{name}_jackal_image_ec2_instance".to_sym)
|
69
|
+
)
|
70
|
+
region region!
|
71
|
+
description opts.fetch(
|
72
|
+
:image_description,
|
73
|
+
"Jackal Generated Image (#{name})"
|
74
|
+
)
|
75
|
+
no_reboot opts.fetch(:no_reboot, true)
|
76
|
+
end
|
77
|
+
end
|
78
|
+
end
|
79
|
+
|
80
|
+
outputs.set!("#{name}_ami_id".to_sym).value attr!("#{name}_jackal_image", :ami_id)
|
81
|
+
|
82
|
+
base_instance
|
83
|
+
|
84
|
+
end
|
@@ -0,0 +1,46 @@
|
|
1
|
+
SparkleFormation.new(:jackal_bus) do
|
2
|
+
|
3
|
+
AWSTemplateFormatVersion '2010-09-09'
|
4
|
+
description 'Jackal CFN Bus'
|
5
|
+
|
6
|
+
dynamic!(:sqs_queue, :jackal)
|
7
|
+
|
8
|
+
dynamic!(:sns_topic, :jackal) do
|
9
|
+
properties.subscription array!(
|
10
|
+
->{
|
11
|
+
endpoint attr!(:jackal_sqs_queue, :arn)
|
12
|
+
protocol 'sqs'
|
13
|
+
}
|
14
|
+
)
|
15
|
+
end
|
16
|
+
|
17
|
+
dynamic!(:sqs_queue_policy, :jackal) do
|
18
|
+
properties do
|
19
|
+
policy_document do
|
20
|
+
version '2012-10-17'
|
21
|
+
id 'jackal-sns-policy'
|
22
|
+
statement array!(
|
23
|
+
->{
|
24
|
+
sid 'jackal-sns-access'
|
25
|
+
effect 'Allow'
|
26
|
+
principal '*'
|
27
|
+
action ['sqs:SendMessage']
|
28
|
+
resource '*'
|
29
|
+
condition.arnEquals.set!('aws:SourceArn', ref!(:jackal_sns_topic))
|
30
|
+
|
31
|
+
}
|
32
|
+
)
|
33
|
+
end
|
34
|
+
queues [ref!(:jackal_sqs_queue)]
|
35
|
+
end
|
36
|
+
end
|
37
|
+
|
38
|
+
outputs do
|
39
|
+
jackal_service_token.value ref!(:jackal_sns_topic)
|
40
|
+
jackal_sns_arn.value ref!(:jackal_sns_topic)
|
41
|
+
jackal_sns_topic.value attr!(:jackal_sns_topic, :topic_name)
|
42
|
+
jackal_sqs_arn.value attr!(:jackal_sqs_queue, :arn)
|
43
|
+
jackal_sqs_http.value ref!(:jackal_sqs_queue)
|
44
|
+
jackal_sqs_queue_name.value attr!(:jackal_sqs_queue, :queue_name)
|
45
|
+
end
|
46
|
+
end
|
@@ -0,0 +1,371 @@
|
|
1
|
+
SparkleFormation.new(:jackal_cfn, :inherit => :jackal_bus) do
|
2
|
+
|
3
|
+
description 'Jackal CFN'
|
4
|
+
|
5
|
+
parameters do
|
6
|
+
jackal_max_cluster_size do
|
7
|
+
type 'Number'
|
8
|
+
value 2
|
9
|
+
end
|
10
|
+
jackal_custom_packages do
|
11
|
+
type 'CommaDelimitedList'
|
12
|
+
default 'bash,jq'
|
13
|
+
end
|
14
|
+
networking_vpc_id do
|
15
|
+
type 'String'
|
16
|
+
default 'none'
|
17
|
+
end
|
18
|
+
networking_subnet_ids do
|
19
|
+
type 'CommaDelimitedList'
|
20
|
+
default 'none'
|
21
|
+
end
|
22
|
+
end
|
23
|
+
|
24
|
+
mappings do
|
25
|
+
config do
|
26
|
+
flavor do
|
27
|
+
classic 'm1.small'
|
28
|
+
vpc 't2.micro'
|
29
|
+
end
|
30
|
+
set!('ap-northeast-1'.disable_camel!) do
|
31
|
+
classic_ami_id 'ami-d5665cbb'
|
32
|
+
vpc_ami_id 'ami-41675d2f'
|
33
|
+
end
|
34
|
+
set!('ap-southeast-1'.disable_camel!) do
|
35
|
+
classic_ami_id 'ami-5a6fa039'
|
36
|
+
vpc_ami_id 'ami-5e6ea13d'
|
37
|
+
end
|
38
|
+
set!('ap-southeast-2'.disable_camel!) do
|
39
|
+
classic_ami_id 'ami-d2d8fcb1'
|
40
|
+
vpc_ami_id 'ami-8cdafeef'
|
41
|
+
end
|
42
|
+
set!('cn-north-1'.disable_camel!) do
|
43
|
+
classic_ami_id 'ami-d97db4b4'
|
44
|
+
vpc_ami_id 'ami-3378b15e'
|
45
|
+
end
|
46
|
+
set!('eu-central-1'.disable_camel!) do
|
47
|
+
classic_ami_id 'ami-15f6ee79'
|
48
|
+
vpc_ami_id 'ami-acf4ecc0'
|
49
|
+
end
|
50
|
+
set!('sa-east-1'.disable_camel!) do
|
51
|
+
classic_ami_id 'ami-6219990e'
|
52
|
+
vpc_ami_id 'ami-1c1b9b70'
|
53
|
+
end
|
54
|
+
set!('eu-west-1'.disable_camel!) do
|
55
|
+
classic_ami_id 'ami-6d70c61e'
|
56
|
+
vpc_ami_id 'ami-bf72c4cc'
|
57
|
+
end
|
58
|
+
set!('us-east-1'.disable_camel!) do
|
59
|
+
classic_ami_id 'ami-59d6f933'
|
60
|
+
vpc_ami_id 'ami-20d3fc4a'
|
61
|
+
end
|
62
|
+
set!('us-west-1'.disable_camel!) do
|
63
|
+
classic_ami_id 'ami-cd2056ad'
|
64
|
+
vpc_ami_id 'ami-842355e4'
|
65
|
+
end
|
66
|
+
set!('us-west-2'.disable_camel!) do
|
67
|
+
classic_ami_id 'ami-a1c721c1'
|
68
|
+
vpc_ami_id 'ami-25c52345'
|
69
|
+
end
|
70
|
+
set!('us-gov-west-1'.disable_camel!) do
|
71
|
+
classic_ami_id 'ami-ecbbd9cf'
|
72
|
+
vpc_ami_id 'ami-d6bbd9f5'
|
73
|
+
end
|
74
|
+
end
|
75
|
+
end
|
76
|
+
|
77
|
+
conditions.vpc_enabled not!(equals!(ref!(:networking_vpc_id), 'none'))
|
78
|
+
|
79
|
+
dynamic!(:ec2_security_group, :jackal) do
|
80
|
+
properties do
|
81
|
+
group_description 'Jackal compute security group'
|
82
|
+
vpc_id if!(:vpc_enabled, ref!(:networking_vpc_id), no_value!)
|
83
|
+
security_group_ingress array!(
|
84
|
+
->{
|
85
|
+
ip_protocol 'tcp'
|
86
|
+
from_port 22
|
87
|
+
to_port 22
|
88
|
+
cidr_ip '0.0.0.0/0'
|
89
|
+
}
|
90
|
+
)
|
91
|
+
end
|
92
|
+
end
|
93
|
+
|
94
|
+
dynamic!(:iam_user, :jackal).properties do
|
95
|
+
path '/'
|
96
|
+
policies array!(
|
97
|
+
->{
|
98
|
+
policy_name 'service-access'
|
99
|
+
policy_document.statement array!(
|
100
|
+
->{
|
101
|
+
effect 'Allow'
|
102
|
+
action '*'
|
103
|
+
resource '*'
|
104
|
+
}
|
105
|
+
)
|
106
|
+
}
|
107
|
+
)
|
108
|
+
end
|
109
|
+
|
110
|
+
dynamic!(:iam_access_key, :jackal).properties.user_name ref!(:jackal_iam_user)
|
111
|
+
|
112
|
+
dynamic!(:jackal_image, :jackal_cfn,
|
113
|
+
:image_id => if!(:vpc_enabled, map!(:config, region!, :vpc_ami_id), map!(:config, region!, :classic_ami_id)),
|
114
|
+
:instance_type => if!(:vpc_enabled, map!(:config, 'Flavor', :vpc), map!(:config, 'Flavor', :classic)),
|
115
|
+
:service_token => ref!(:jackal_sns_topic)
|
116
|
+
) do
|
117
|
+
properties do
|
118
|
+
security_groups [ref!(:jackal_ec2_security_group)]
|
119
|
+
network_interfaces if!(
|
120
|
+
:vpc_enabled,
|
121
|
+
array!(
|
122
|
+
-> {
|
123
|
+
device_index 0
|
124
|
+
associate_public_ip_address 'true'
|
125
|
+
subnet_id select!(0, ref!(:networking_subnet_ids))
|
126
|
+
}
|
127
|
+
),
|
128
|
+
[]
|
129
|
+
)
|
130
|
+
end
|
131
|
+
metadata('AWS::CloudFormation::Init') do
|
132
|
+
camel_keys_set!(:auto_disable)
|
133
|
+
config do
|
134
|
+
files('/etc/jackal/configuration.json') do
|
135
|
+
content do
|
136
|
+
jackal do
|
137
|
+
require [
|
138
|
+
'carnivore-sqs',
|
139
|
+
'jackal-cfn'
|
140
|
+
]
|
141
|
+
cfn do
|
142
|
+
sources do
|
143
|
+
input do
|
144
|
+
type :sqs
|
145
|
+
args do
|
146
|
+
fog do
|
147
|
+
aws_access_key_id ref!(:jackal_iam_access_key)
|
148
|
+
aws_secret_access_key attr!(:jackal_iam_access_key, :secret_access_key)
|
149
|
+
region region!
|
150
|
+
end
|
151
|
+
queues [
|
152
|
+
ref!(:jackal_sqs_queue)
|
153
|
+
]
|
154
|
+
end
|
155
|
+
end
|
156
|
+
end
|
157
|
+
config do
|
158
|
+
reprocess true
|
159
|
+
ami.credentials.compute do
|
160
|
+
aws_access_key_id ref!(:jackal_iam_access_key)
|
161
|
+
aws_secret_access_key attr!(:jackal_iam_access_key, :secret_access_key)
|
162
|
+
region region!
|
163
|
+
end
|
164
|
+
jackal_stack do
|
165
|
+
credentials do
|
166
|
+
storage do
|
167
|
+
aws_access_key_id ref!(:jackal_iam_access_key)
|
168
|
+
aws_secret_access_key attr!(:jackal_iam_access_key, :secret_access_key)
|
169
|
+
end
|
170
|
+
us_west_1 do
|
171
|
+
provider "aws"
|
172
|
+
aws_access_key_id ref!(:jackal_iam_access_key)
|
173
|
+
aws_secret_access_key attr!(:jackal_iam_access_key, :secret_access_key)
|
174
|
+
aws_region 'us-west-1'
|
175
|
+
end
|
176
|
+
end
|
177
|
+
end
|
178
|
+
end
|
179
|
+
callbacks [
|
180
|
+
'Jackal::Cfn::Resource',
|
181
|
+
'Jackal::Cfn::AmiRegister',
|
182
|
+
'Jackal::Cfn::AmiManager',
|
183
|
+
'Jackal::Cfn::JackalStack',
|
184
|
+
'Jackal::Cfn::OrchestrationUnit'
|
185
|
+
]
|
186
|
+
end
|
187
|
+
end
|
188
|
+
end
|
189
|
+
end
|
190
|
+
files('/etc/init.d/jackal') do
|
191
|
+
content "#!/bin/sh\nstart-stop-daemon --start --oknodo --user jackal --pidfile /var/run/jackal.pid --make-pidfile --chuid jackal --background --exec /bin/bash -- -c 'jackal -c /etc/jackal > /opt/jackal/run.log 2>&1'\n"
|
192
|
+
mode '000755'
|
193
|
+
end
|
194
|
+
files('/jackal-example.json') do
|
195
|
+
content.set!('OriginStack', stack_id!)
|
196
|
+
end
|
197
|
+
users.jackal.homeDir '/opt/jackal'
|
198
|
+
commands('00_ntp_sync') do
|
199
|
+
command 'apt-get install ntpdate -qy && ntpdate -b -s pool.ntp.org'
|
200
|
+
end
|
201
|
+
commands('01_jackal_directory') do
|
202
|
+
command 'mkdir /opt/jackal && chown jackal /opt/jackal'
|
203
|
+
end
|
204
|
+
commands('02_custom_packages') do
|
205
|
+
command join!(
|
206
|
+
'apt-get install -yq ',
|
207
|
+
join!(
|
208
|
+
ref!(:jackal_custom_packages),
|
209
|
+
:options => {
|
210
|
+
:delimiter => ' '
|
211
|
+
}
|
212
|
+
)
|
213
|
+
)
|
214
|
+
ignoreErrors 'true'
|
215
|
+
end
|
216
|
+
commands('02_software_properties_old') do
|
217
|
+
command 'apt-get install -yq python-software-properties'
|
218
|
+
ignoreErrors 'true'
|
219
|
+
end
|
220
|
+
commands('02_software_properties_new') do
|
221
|
+
command 'apt-get install -yq software-properties-common'
|
222
|
+
ignoreErrors 'true'
|
223
|
+
end
|
224
|
+
commands('03_add_repository') do
|
225
|
+
command 'apt-add-repository ppa:brightbox/ruby-ng -y'
|
226
|
+
end
|
227
|
+
commands('04_apt_reupdate') do
|
228
|
+
command 'apt-get update'
|
229
|
+
end
|
230
|
+
commands('05_ruby_install') do
|
231
|
+
command 'apt-get install -qy ruby2.2 ruby2.2-dev libyajl-dev build-essential libcurl3-dev libxslt1-dev libxml2 zlib1g-dev awscli'
|
232
|
+
end
|
233
|
+
commands('06_jackal_install') do
|
234
|
+
command 'gem install --no-document jackal-cfn carnivore-sqs'
|
235
|
+
end
|
236
|
+
commands('07_jackal_startup') do
|
237
|
+
command '/etc/init.d/jackal start'
|
238
|
+
end
|
239
|
+
commands('08_pause_for_file_stabilization') do
|
240
|
+
command 'sleep 180'
|
241
|
+
end
|
242
|
+
commands('09_notify_complete') do
|
243
|
+
command join!(
|
244
|
+
"cfn-signal -e 0 -r 'Provision complete' --resource #{resource_name!} --region ",
|
245
|
+
region!,
|
246
|
+
' --stack ',
|
247
|
+
stack_name!
|
248
|
+
)
|
249
|
+
end
|
250
|
+
commands('10_pause_for_dramatic_effect') do
|
251
|
+
command 'sleep 3500'
|
252
|
+
end
|
253
|
+
commands('11_kill_self') do
|
254
|
+
command join!(
|
255
|
+
'aws ec2 terminate-instances --instance-ids `curl -s http://169.254.169.254/latest/meta-data/instance-id` --region ',
|
256
|
+
region!
|
257
|
+
)
|
258
|
+
env do
|
259
|
+
set!('AWS_ACCESS_KEY_ID', ref!(:jackal_iam_access_key))
|
260
|
+
set!('AWS_SECRET_ACCESS_KEY', attr!(:jackal_iam_access_key, :secret_access_key))
|
261
|
+
end
|
262
|
+
end
|
263
|
+
end
|
264
|
+
end
|
265
|
+
end
|
266
|
+
|
267
|
+
dynamic!(:auto_scaling_launch_configuration, :jackal) do
|
268
|
+
properties do
|
269
|
+
image_id attr!(:jackal_cfn_jackal_image, :ami_id)
|
270
|
+
key_name ref!(:jackal_cfn_key_name)
|
271
|
+
instance_type if!(:vpc_enabled, map!(:config, 'Flavor', :vpc), map!(:config, 'Flavor', :classic))
|
272
|
+
security_groups [ref!(:jackal_ec2_security_group)]
|
273
|
+
user_data base64!(
|
274
|
+
join!(
|
275
|
+
"#!/bin/bash\n",
|
276
|
+
'/usr/local/bin/cfn-init -v --region ',
|
277
|
+
region!,
|
278
|
+
' -s ',
|
279
|
+
stack_name!,
|
280
|
+
" -r #{resource_name!}"
|
281
|
+
)
|
282
|
+
)
|
283
|
+
end
|
284
|
+
metadata('AWS::CloudFormation::Init') do
|
285
|
+
camel_keys_set!(:auto_disable)
|
286
|
+
config do
|
287
|
+
files('/usr/local/bin/suicider') do
|
288
|
+
content join!(
|
289
|
+
"#!/bin/bash\n",
|
290
|
+
"while true\ndo\nsleep 3500\n",
|
291
|
+
"tail -n 1 /opt/jackal/run.log | grep 'no message received'\n",
|
292
|
+
"if [ $? -eq 0 ]\nthen\n",
|
293
|
+
'AWS_ACCESS_KEY_ID="',
|
294
|
+
ref!(:jackal_iam_access_key),
|
295
|
+
'" AWS_SECRET_ACCESS_KEY="',
|
296
|
+
attr!(:jackal_iam_access_key, :secret_access_key),
|
297
|
+
'" aws autoscaling terminate-instance-in-autoscaling-group --should-decrement-desired-capacity --instance-id `curl -s http://169.254.169.254/latest/meta-data/instance-id` --region ',
|
298
|
+
region!,
|
299
|
+
"\nexit 0\nfi\ndone\n"
|
300
|
+
)
|
301
|
+
mode '000700'
|
302
|
+
end
|
303
|
+
commands('00_jackal_init') do
|
304
|
+
command '/etc/init.d/jackal start'
|
305
|
+
end
|
306
|
+
commands('01_suicider') do
|
307
|
+
command 'nohup bash -c /usr/local/bin/suicider &'
|
308
|
+
end
|
309
|
+
end
|
310
|
+
end
|
311
|
+
end
|
312
|
+
|
313
|
+
dynamic!(:auto_scaling_group, :jackal) do
|
314
|
+
properties do
|
315
|
+
cooldown 500
|
316
|
+
max_size ref!(:jackal_max_cluster_size)
|
317
|
+
min_size 0
|
318
|
+
availability_zones if!(:vpc_enabled, no_value!, azs!)
|
319
|
+
VPCZoneIdentifier if!(:vpc_enabled, ref!(:networking_subnet_ids), no_value!)
|
320
|
+
launch_configuration_name ref!(:jackal_auto_scaling_launch_configuration)
|
321
|
+
end
|
322
|
+
end
|
323
|
+
|
324
|
+
dynamic!(:auto_scaling_scaling_policy, :jackal) do
|
325
|
+
properties do
|
326
|
+
adjustment_type 'ChangeInCapacity'
|
327
|
+
auto_scaling_group_name ref!(:jackal_auto_scaling_group)
|
328
|
+
cooldown 600
|
329
|
+
scaling_adjustment 1
|
330
|
+
end
|
331
|
+
end
|
332
|
+
|
333
|
+
dynamic!(:cloud_watch_alarm, :jackal) do
|
334
|
+
properties do
|
335
|
+
actions_enabled true
|
336
|
+
alarm_actions [ref!(:jackal_auto_scaling_scaling_policy)]
|
337
|
+
alarm_description 'Jackal Kickstart'
|
338
|
+
comparison_operator 'GreaterThanOrEqualToThreshold'
|
339
|
+
evaluation_periods 2
|
340
|
+
metric_name 'ApproximateNumberOfMessagesVisible'
|
341
|
+
namespace 'AWS/SQS'
|
342
|
+
dimensions array!(
|
343
|
+
->{
|
344
|
+
name 'QueueName'
|
345
|
+
value attr!(:jackal_sqs_queue, :queue_name)
|
346
|
+
}
|
347
|
+
)
|
348
|
+
period 60
|
349
|
+
statistic 'Average'
|
350
|
+
threshold 1
|
351
|
+
end
|
352
|
+
end
|
353
|
+
|
354
|
+
resources.jackal_image_cleanup do
|
355
|
+
type 'Custom::AmiManager'
|
356
|
+
depends_on!(
|
357
|
+
:jackal_cloud_watch_alarm,
|
358
|
+
:jackal_sqs_queue_policy,
|
359
|
+
:jackal_auto_scaling_scaling_policy,
|
360
|
+
:jackal_auto_scaling_group
|
361
|
+
)
|
362
|
+
properties do
|
363
|
+
service_token ref!(:jackal_sns_topic)
|
364
|
+
parameters do
|
365
|
+
ami_id attr!(:jackal_cfn_jackal_image, :ami_id)
|
366
|
+
region region!
|
367
|
+
end
|
368
|
+
end
|
369
|
+
end
|
370
|
+
|
371
|
+
end
|
@@ -0,0 +1,15 @@
|
|
1
|
+
SparkleFormation.new(:jackal_cfn_usage_example) do
|
2
|
+
|
3
|
+
AWSTemplateFormatVersion '2010-09-09'
|
4
|
+
description 'Jackal CFN Example Usage'
|
5
|
+
|
6
|
+
parameters.jackal_service_token.type 'String'
|
7
|
+
|
8
|
+
dynamic!(:orchestration_unit, :example).properties.parameters.exec 'cat /jackal-example.json'
|
9
|
+
|
10
|
+
outputs do
|
11
|
+
full_result.value attr!(:example_orchestration_unit, :orchestration_unit_result)
|
12
|
+
key_result.value attr!(:example_orchestration_unit, :origin_stack)
|
13
|
+
end
|
14
|
+
|
15
|
+
end
|
@@ -0,0 +1,13 @@
|
|
1
|
+
Gem::Specification.new do |s|
|
2
|
+
s.name = 'sparkle-pack-jackal-cfn'
|
3
|
+
s.version = '0.1.0'
|
4
|
+
s.summary = 'Jackal CFN compute stacker'
|
5
|
+
s.author = 'Chris Roberts'
|
6
|
+
s.email = 'chrisroberts.code@gmail.com'
|
7
|
+
s.homepage = 'http://github.com/carnivore-rb/sparkle-pack-jackal-cfn'
|
8
|
+
s.description = 'SparkleFormation pack for generating Jackal CFN compute stack'
|
9
|
+
s.license = 'Apache-2.0'
|
10
|
+
s.require_path = 'lib'
|
11
|
+
s.add_runtime_dependency 'sparkle_formation', '>= 2.1.0'
|
12
|
+
s.files = Dir['{lib,docs}/**/*'] + %w(sparkle-pack-jackal-cfn.gemspec README.md CHANGELOG.md LICENSE)
|
13
|
+
end
|
metadata
ADDED
@@ -0,0 +1,72 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: sparkle-pack-jackal-cfn
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 0.1.0
|
5
|
+
platform: ruby
|
6
|
+
authors:
|
7
|
+
- Chris Roberts
|
8
|
+
autorequire:
|
9
|
+
bindir: bin
|
10
|
+
cert_chain: []
|
11
|
+
date: 2016-02-08 00:00:00.000000000 Z
|
12
|
+
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
name: sparkle_formation
|
15
|
+
requirement: !ruby/object:Gem::Requirement
|
16
|
+
requirements:
|
17
|
+
- - ">="
|
18
|
+
- !ruby/object:Gem::Version
|
19
|
+
version: 2.1.0
|
20
|
+
type: :runtime
|
21
|
+
prerelease: false
|
22
|
+
version_requirements: !ruby/object:Gem::Requirement
|
23
|
+
requirements:
|
24
|
+
- - ">="
|
25
|
+
- !ruby/object:Gem::Version
|
26
|
+
version: 2.1.0
|
27
|
+
description: SparkleFormation pack for generating Jackal CFN compute stack
|
28
|
+
email: chrisroberts.code@gmail.com
|
29
|
+
executables: []
|
30
|
+
extensions: []
|
31
|
+
extra_rdoc_files: []
|
32
|
+
files:
|
33
|
+
- CHANGELOG.md
|
34
|
+
- LICENSE
|
35
|
+
- README.md
|
36
|
+
- lib/sparkle-pack-jackal-cfn.rb
|
37
|
+
- lib/sparkleformation/dynamics/ami_manager.rb
|
38
|
+
- lib/sparkleformation/dynamics/ami_register.rb
|
39
|
+
- lib/sparkleformation/dynamics/hash_extractor.rb
|
40
|
+
- lib/sparkleformation/dynamics/jackal_image.rb
|
41
|
+
- lib/sparkleformation/dynamics/jackal_stack.rb
|
42
|
+
- lib/sparkleformation/dynamics/orchestration_unit.rb
|
43
|
+
- lib/sparkleformation/jackal_bus.rb
|
44
|
+
- lib/sparkleformation/jackal_cfn.rb
|
45
|
+
- lib/sparkleformation/jackal_usage_example.rb
|
46
|
+
- sparkle-pack-jackal-cfn.gemspec
|
47
|
+
homepage: http://github.com/carnivore-rb/sparkle-pack-jackal-cfn
|
48
|
+
licenses:
|
49
|
+
- Apache-2.0
|
50
|
+
metadata: {}
|
51
|
+
post_install_message:
|
52
|
+
rdoc_options: []
|
53
|
+
require_paths:
|
54
|
+
- lib
|
55
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
56
|
+
requirements:
|
57
|
+
- - ">="
|
58
|
+
- !ruby/object:Gem::Version
|
59
|
+
version: '0'
|
60
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
61
|
+
requirements:
|
62
|
+
- - ">="
|
63
|
+
- !ruby/object:Gem::Version
|
64
|
+
version: '0'
|
65
|
+
requirements: []
|
66
|
+
rubyforge_project:
|
67
|
+
rubygems_version: 2.4.8
|
68
|
+
signing_key:
|
69
|
+
specification_version: 4
|
70
|
+
summary: Jackal CFN compute stacker
|
71
|
+
test_files: []
|
72
|
+
has_rdoc:
|