sparkle-pack-jackal-cfn 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +2 -0
- data/LICENSE +13 -0
- data/README.md +129 -0
- data/lib/sparkle-pack-jackal-cfn.rb +3 -0
- data/lib/sparkleformation/dynamics/ami_manager.rb +8 -0
- data/lib/sparkleformation/dynamics/ami_register.rb +8 -0
- data/lib/sparkleformation/dynamics/hash_extractor.rb +8 -0
- data/lib/sparkleformation/dynamics/jackal_image.rb +84 -0
- data/lib/sparkleformation/dynamics/jackal_stack.rb +8 -0
- data/lib/sparkleformation/dynamics/orchestration_unit.rb +8 -0
- data/lib/sparkleformation/jackal_bus.rb +46 -0
- data/lib/sparkleformation/jackal_cfn.rb +371 -0
- data/lib/sparkleformation/jackal_usage_example.rb +15 -0
- data/sparkle-pack-jackal-cfn.gemspec +13 -0
- metadata +72 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA1:
|
|
3
|
+
metadata.gz: 68eac1e8d59b0d0c7ce62f3c8e467313e8c50bf6
|
|
4
|
+
data.tar.gz: 73e0fb3a5760fd6a5eaf1ff1a82f3c3f6d9d9f5a
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: 56b6ba743aff42df8bf5ff15b67fea0e8cbecf162f93de577c7cfd58d13406cd8fac42fa0b7140b7de53416e6ad695827f2a29200cdfa35455fe29c9c09c8760
|
|
7
|
+
data.tar.gz: 7e685a95dcf3b686bbf41e455d5f5962d05a755bea52e3e0f9fc1cd9b5efadfcac9301cd328e4700188e61e495c793d2d079520cc77811fc96262ea5f3ead498
|
data/CHANGELOG.md
ADDED
data/LICENSE
ADDED
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
Copyright 2016 Chris Roberts
|
|
2
|
+
|
|
3
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
|
4
|
+
you may not use this file except in compliance with the License.
|
|
5
|
+
You may obtain a copy of the License at
|
|
6
|
+
|
|
7
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
|
8
|
+
|
|
9
|
+
Unless required by applicable law or agreed to in writing, software
|
|
10
|
+
distributed under the License is distributed on an "AS IS" BASIS,
|
|
11
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
12
|
+
See the License for the specific language governing permissions and
|
|
13
|
+
limitations under the License.
|
data/README.md
ADDED
|
@@ -0,0 +1,129 @@
|
|
|
1
|
+
# SparklePack - Jackal CFN
|
|
2
|
+
|
|
3
|
+
This is a [SparkleFormation][1] [SparklePack][2] for [jackal-cfn][3]. It provides
|
|
4
|
+
integrations for the custom resources provided by the [jackal-cfn][3]
|
|
5
|
+
library as well as templates for worker setup and example usage.
|
|
6
|
+
|
|
7
|
+
## Integrations
|
|
8
|
+
|
|
9
|
+
### Dynamics
|
|
10
|
+
|
|
11
|
+
This [SparklePack][2] provides dynamics for custom resources provided
|
|
12
|
+
by [jackal-cfn][3]:
|
|
13
|
+
|
|
14
|
+
* `:ami_manager`
|
|
15
|
+
* `:ami_register`
|
|
16
|
+
* `:hash_extractor`
|
|
17
|
+
* `:jackal_stack`
|
|
18
|
+
* `:orchestration_unit`
|
|
19
|
+
|
|
20
|
+
It also provides a customized `:jackal_image` dynamic which will
|
|
21
|
+
create the resource as well as an output of the newly created
|
|
22
|
+
AMI ID.
|
|
23
|
+
|
|
24
|
+
### Templates
|
|
25
|
+
|
|
26
|
+
This [SparklePack][2] includes 3 templates:
|
|
27
|
+
|
|
28
|
+
1. `jackal_cfn` - Creates a working processor to handle requests
|
|
29
|
+
2. `jackal_bus` - Creates _only_ the required message bus resources
|
|
30
|
+
3. `jackal_usage_example` - Example template to display usage of the OrchestrationUnit resource
|
|
31
|
+
|
|
32
|
+
_NOTE: The `jackal_cfn` template only supports the follow regions:_
|
|
33
|
+
|
|
34
|
+
* ap-northeast-1
|
|
35
|
+
* ap-southeast-1
|
|
36
|
+
* ap-southeast-2
|
|
37
|
+
* cn-north-1
|
|
38
|
+
* eu-central-1
|
|
39
|
+
* eu-west-1
|
|
40
|
+
* sa-east-1
|
|
41
|
+
* us-east-1
|
|
42
|
+
* us-west-1
|
|
43
|
+
* us-west-2
|
|
44
|
+
* us-gov-west-1
|
|
45
|
+
|
|
46
|
+
## Usage
|
|
47
|
+
|
|
48
|
+
_NOTE: Assumes a working `sfn` setup. (see: [Getting started guide][4])_
|
|
49
|
+
|
|
50
|
+
### Setup
|
|
51
|
+
|
|
52
|
+
First, include the [SparklePack][2] into the bundle by adding this line to
|
|
53
|
+
the `./Gemfile`:
|
|
54
|
+
|
|
55
|
+
```ruby
|
|
56
|
+
gem 'sparkle-pack-jackal-cfn'
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
Next, update the local bundle:
|
|
60
|
+
|
|
61
|
+
```
|
|
62
|
+
$ bundle update
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
Now enable the [SparklePack][2] within the `.sfn` configuration file:
|
|
66
|
+
|
|
67
|
+
```ruby
|
|
68
|
+
Configuration.new do
|
|
69
|
+
...
|
|
70
|
+
sparkle_pack 'sparkle-pack-jackal-cfn'
|
|
71
|
+
...
|
|
72
|
+
end
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
### Build a jackal-cfn processor
|
|
76
|
+
|
|
77
|
+
To use the [jackal-cfn][3] resources, a processor instance must be available. The
|
|
78
|
+
[SparklePack][2] provides a template to build a processor stack. This stack will
|
|
79
|
+
build a single EC2 instance which will be used as the base for an on demand
|
|
80
|
+
autoscaling group. To create the stack:
|
|
81
|
+
|
|
82
|
+
```
|
|
83
|
+
$ bundle exec sfn create STACK_NAME_PROCESSOR --file jackal_cfn
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
### Using the jackal-cfn processor
|
|
87
|
+
|
|
88
|
+
This [SparklePack][2] includes an example template that uses the `OrchestrationUnit`
|
|
89
|
+
resource to read the contents of a file on the running processor instance. The
|
|
90
|
+
contents of the file is a JSON serialized Hash that includes the ID of the
|
|
91
|
+
current stack. The example stack will return the full result as well as the
|
|
92
|
+
specific `"OriginStack"` value in the outputs:
|
|
93
|
+
|
|
94
|
+
```
|
|
95
|
+
$ bundle exec sfn create STACK_NAME_EXAMPLE --file jackal_usage_example --apply-stack STACK_NAME_PROCESSOR
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
### All in one
|
|
99
|
+
|
|
100
|
+
The templates in this [SparklePack][2] support nesting. To create a single stack
|
|
101
|
+
that includes the `jackal_cfn` template and the `jackal_usage_example` template,
|
|
102
|
+
create a new local template:
|
|
103
|
+
|
|
104
|
+
```ruby
|
|
105
|
+
# ./sparkleformation/full_jackal.rb
|
|
106
|
+
|
|
107
|
+
SparkleFormation.new(:full_jackal) do
|
|
108
|
+
nest!(:jackal_cfn)
|
|
109
|
+
nest!(:jackal_usage_example)
|
|
110
|
+
end
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
And create the nested stacks:
|
|
114
|
+
|
|
115
|
+
```
|
|
116
|
+
$ bundle exec sfn create STACK_NAME_ALL --file full_stack
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
## Info
|
|
120
|
+
|
|
121
|
+
* Repository: https://github.com/carnivore-rb/sparkle-pack-jackal-cfn
|
|
122
|
+
* Jackal CFN: https://github.com/carnivore-rb/jackal-cfn
|
|
123
|
+
* SparkleFormation: http://www.sparkleformation.io
|
|
124
|
+
* IRC: Freenode @ #carnivore
|
|
125
|
+
|
|
126
|
+
[1]: http://www.sparkleformation.io/
|
|
127
|
+
[2]: http://www.sparkleformation.io/docs/sparkle_formation/sparkle-packs.html
|
|
128
|
+
[3]: https://github.com/carnivore-rb/jackal-cfn
|
|
129
|
+
[4]: http://www.sparkleformation.io/docs/guides/getting-started.html
|
|
@@ -0,0 +1,84 @@
|
|
|
1
|
+
SparkleFormation.dynamic(:jackal_image) do |name, opts={}|
|
|
2
|
+
|
|
3
|
+
ec2_required_info = [:image_id, :key_name, :instance_type]
|
|
4
|
+
|
|
5
|
+
ec2_required_info.each do |req_value|
|
|
6
|
+
unless(opts[req_value])
|
|
7
|
+
parameters.set!("#{name}_#{req_value}".to_sym).type 'String'
|
|
8
|
+
end
|
|
9
|
+
end
|
|
10
|
+
|
|
11
|
+
unless(opts[:service_token])
|
|
12
|
+
parameters.jackal_service_token.type 'String'
|
|
13
|
+
end
|
|
14
|
+
|
|
15
|
+
base_instance = dynamic!(:ec2_instance, "#{name}_jackal_image".to_sym) do
|
|
16
|
+
properties do
|
|
17
|
+
ec2_required_info.each do |req_value|
|
|
18
|
+
set!(req_value, opts.fetch(req_value, ref!("#{name}_#{req_value}".to_sym)))
|
|
19
|
+
end
|
|
20
|
+
if(opts[:user_data])
|
|
21
|
+
user_data opts[:user_data]
|
|
22
|
+
else
|
|
23
|
+
user_data base64!(
|
|
24
|
+
join!(
|
|
25
|
+
"#!/bin/bash\n",
|
|
26
|
+
"apt-get update\n",
|
|
27
|
+
"apt-get -y install python-setuptools\n",
|
|
28
|
+
"easy_install https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz\n",
|
|
29
|
+
'/usr/local/bin/cfn-init -v --region ',
|
|
30
|
+
region!,
|
|
31
|
+
' -s ',
|
|
32
|
+
stack_name!,
|
|
33
|
+
" -r #{resource_name!}\n",
|
|
34
|
+
"cfn-signal -e $? -r 'Provision complete' --resource #{resource_name!} --region ",
|
|
35
|
+
region!,
|
|
36
|
+
' --stack ',
|
|
37
|
+
stack_name!
|
|
38
|
+
)
|
|
39
|
+
)
|
|
40
|
+
end
|
|
41
|
+
end
|
|
42
|
+
unless(opts[:user_data])
|
|
43
|
+
creation_policy.resource_signal do
|
|
44
|
+
count 1
|
|
45
|
+
timeout 'PT30M'
|
|
46
|
+
end
|
|
47
|
+
end
|
|
48
|
+
end
|
|
49
|
+
|
|
50
|
+
resources.set!("#{name}_jackal_image".to_sym) do
|
|
51
|
+
type 'Custom::AmiRegister'
|
|
52
|
+
properties do
|
|
53
|
+
service_token opts.fetch(:service_token, ref!(:jackal_service_token))
|
|
54
|
+
parameters do
|
|
55
|
+
name join!(
|
|
56
|
+
"jackal-image-#{name}",
|
|
57
|
+
opts.fetch(
|
|
58
|
+
:image_instance_id,
|
|
59
|
+
ref!("#{name}_jackal_image_ec2_instance".to_sym)
|
|
60
|
+
),
|
|
61
|
+
stack_name!,
|
|
62
|
+
:options => {
|
|
63
|
+
:delimiter => '-'
|
|
64
|
+
}
|
|
65
|
+
)
|
|
66
|
+
instance_id opts.fetch(
|
|
67
|
+
:image_instance_id,
|
|
68
|
+
ref!("#{name}_jackal_image_ec2_instance".to_sym)
|
|
69
|
+
)
|
|
70
|
+
region region!
|
|
71
|
+
description opts.fetch(
|
|
72
|
+
:image_description,
|
|
73
|
+
"Jackal Generated Image (#{name})"
|
|
74
|
+
)
|
|
75
|
+
no_reboot opts.fetch(:no_reboot, true)
|
|
76
|
+
end
|
|
77
|
+
end
|
|
78
|
+
end
|
|
79
|
+
|
|
80
|
+
outputs.set!("#{name}_ami_id".to_sym).value attr!("#{name}_jackal_image", :ami_id)
|
|
81
|
+
|
|
82
|
+
base_instance
|
|
83
|
+
|
|
84
|
+
end
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
SparkleFormation.new(:jackal_bus) do
|
|
2
|
+
|
|
3
|
+
AWSTemplateFormatVersion '2010-09-09'
|
|
4
|
+
description 'Jackal CFN Bus'
|
|
5
|
+
|
|
6
|
+
dynamic!(:sqs_queue, :jackal)
|
|
7
|
+
|
|
8
|
+
dynamic!(:sns_topic, :jackal) do
|
|
9
|
+
properties.subscription array!(
|
|
10
|
+
->{
|
|
11
|
+
endpoint attr!(:jackal_sqs_queue, :arn)
|
|
12
|
+
protocol 'sqs'
|
|
13
|
+
}
|
|
14
|
+
)
|
|
15
|
+
end
|
|
16
|
+
|
|
17
|
+
dynamic!(:sqs_queue_policy, :jackal) do
|
|
18
|
+
properties do
|
|
19
|
+
policy_document do
|
|
20
|
+
version '2012-10-17'
|
|
21
|
+
id 'jackal-sns-policy'
|
|
22
|
+
statement array!(
|
|
23
|
+
->{
|
|
24
|
+
sid 'jackal-sns-access'
|
|
25
|
+
effect 'Allow'
|
|
26
|
+
principal '*'
|
|
27
|
+
action ['sqs:SendMessage']
|
|
28
|
+
resource '*'
|
|
29
|
+
condition.arnEquals.set!('aws:SourceArn', ref!(:jackal_sns_topic))
|
|
30
|
+
|
|
31
|
+
}
|
|
32
|
+
)
|
|
33
|
+
end
|
|
34
|
+
queues [ref!(:jackal_sqs_queue)]
|
|
35
|
+
end
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
outputs do
|
|
39
|
+
jackal_service_token.value ref!(:jackal_sns_topic)
|
|
40
|
+
jackal_sns_arn.value ref!(:jackal_sns_topic)
|
|
41
|
+
jackal_sns_topic.value attr!(:jackal_sns_topic, :topic_name)
|
|
42
|
+
jackal_sqs_arn.value attr!(:jackal_sqs_queue, :arn)
|
|
43
|
+
jackal_sqs_http.value ref!(:jackal_sqs_queue)
|
|
44
|
+
jackal_sqs_queue_name.value attr!(:jackal_sqs_queue, :queue_name)
|
|
45
|
+
end
|
|
46
|
+
end
|
|
@@ -0,0 +1,371 @@
|
|
|
1
|
+
SparkleFormation.new(:jackal_cfn, :inherit => :jackal_bus) do
|
|
2
|
+
|
|
3
|
+
description 'Jackal CFN'
|
|
4
|
+
|
|
5
|
+
parameters do
|
|
6
|
+
jackal_max_cluster_size do
|
|
7
|
+
type 'Number'
|
|
8
|
+
value 2
|
|
9
|
+
end
|
|
10
|
+
jackal_custom_packages do
|
|
11
|
+
type 'CommaDelimitedList'
|
|
12
|
+
default 'bash,jq'
|
|
13
|
+
end
|
|
14
|
+
networking_vpc_id do
|
|
15
|
+
type 'String'
|
|
16
|
+
default 'none'
|
|
17
|
+
end
|
|
18
|
+
networking_subnet_ids do
|
|
19
|
+
type 'CommaDelimitedList'
|
|
20
|
+
default 'none'
|
|
21
|
+
end
|
|
22
|
+
end
|
|
23
|
+
|
|
24
|
+
mappings do
|
|
25
|
+
config do
|
|
26
|
+
flavor do
|
|
27
|
+
classic 'm1.small'
|
|
28
|
+
vpc 't2.micro'
|
|
29
|
+
end
|
|
30
|
+
set!('ap-northeast-1'.disable_camel!) do
|
|
31
|
+
classic_ami_id 'ami-d5665cbb'
|
|
32
|
+
vpc_ami_id 'ami-41675d2f'
|
|
33
|
+
end
|
|
34
|
+
set!('ap-southeast-1'.disable_camel!) do
|
|
35
|
+
classic_ami_id 'ami-5a6fa039'
|
|
36
|
+
vpc_ami_id 'ami-5e6ea13d'
|
|
37
|
+
end
|
|
38
|
+
set!('ap-southeast-2'.disable_camel!) do
|
|
39
|
+
classic_ami_id 'ami-d2d8fcb1'
|
|
40
|
+
vpc_ami_id 'ami-8cdafeef'
|
|
41
|
+
end
|
|
42
|
+
set!('cn-north-1'.disable_camel!) do
|
|
43
|
+
classic_ami_id 'ami-d97db4b4'
|
|
44
|
+
vpc_ami_id 'ami-3378b15e'
|
|
45
|
+
end
|
|
46
|
+
set!('eu-central-1'.disable_camel!) do
|
|
47
|
+
classic_ami_id 'ami-15f6ee79'
|
|
48
|
+
vpc_ami_id 'ami-acf4ecc0'
|
|
49
|
+
end
|
|
50
|
+
set!('sa-east-1'.disable_camel!) do
|
|
51
|
+
classic_ami_id 'ami-6219990e'
|
|
52
|
+
vpc_ami_id 'ami-1c1b9b70'
|
|
53
|
+
end
|
|
54
|
+
set!('eu-west-1'.disable_camel!) do
|
|
55
|
+
classic_ami_id 'ami-6d70c61e'
|
|
56
|
+
vpc_ami_id 'ami-bf72c4cc'
|
|
57
|
+
end
|
|
58
|
+
set!('us-east-1'.disable_camel!) do
|
|
59
|
+
classic_ami_id 'ami-59d6f933'
|
|
60
|
+
vpc_ami_id 'ami-20d3fc4a'
|
|
61
|
+
end
|
|
62
|
+
set!('us-west-1'.disable_camel!) do
|
|
63
|
+
classic_ami_id 'ami-cd2056ad'
|
|
64
|
+
vpc_ami_id 'ami-842355e4'
|
|
65
|
+
end
|
|
66
|
+
set!('us-west-2'.disable_camel!) do
|
|
67
|
+
classic_ami_id 'ami-a1c721c1'
|
|
68
|
+
vpc_ami_id 'ami-25c52345'
|
|
69
|
+
end
|
|
70
|
+
set!('us-gov-west-1'.disable_camel!) do
|
|
71
|
+
classic_ami_id 'ami-ecbbd9cf'
|
|
72
|
+
vpc_ami_id 'ami-d6bbd9f5'
|
|
73
|
+
end
|
|
74
|
+
end
|
|
75
|
+
end
|
|
76
|
+
|
|
77
|
+
conditions.vpc_enabled not!(equals!(ref!(:networking_vpc_id), 'none'))
|
|
78
|
+
|
|
79
|
+
dynamic!(:ec2_security_group, :jackal) do
|
|
80
|
+
properties do
|
|
81
|
+
group_description 'Jackal compute security group'
|
|
82
|
+
vpc_id if!(:vpc_enabled, ref!(:networking_vpc_id), no_value!)
|
|
83
|
+
security_group_ingress array!(
|
|
84
|
+
->{
|
|
85
|
+
ip_protocol 'tcp'
|
|
86
|
+
from_port 22
|
|
87
|
+
to_port 22
|
|
88
|
+
cidr_ip '0.0.0.0/0'
|
|
89
|
+
}
|
|
90
|
+
)
|
|
91
|
+
end
|
|
92
|
+
end
|
|
93
|
+
|
|
94
|
+
dynamic!(:iam_user, :jackal).properties do
|
|
95
|
+
path '/'
|
|
96
|
+
policies array!(
|
|
97
|
+
->{
|
|
98
|
+
policy_name 'service-access'
|
|
99
|
+
policy_document.statement array!(
|
|
100
|
+
->{
|
|
101
|
+
effect 'Allow'
|
|
102
|
+
action '*'
|
|
103
|
+
resource '*'
|
|
104
|
+
}
|
|
105
|
+
)
|
|
106
|
+
}
|
|
107
|
+
)
|
|
108
|
+
end
|
|
109
|
+
|
|
110
|
+
dynamic!(:iam_access_key, :jackal).properties.user_name ref!(:jackal_iam_user)
|
|
111
|
+
|
|
112
|
+
dynamic!(:jackal_image, :jackal_cfn,
|
|
113
|
+
:image_id => if!(:vpc_enabled, map!(:config, region!, :vpc_ami_id), map!(:config, region!, :classic_ami_id)),
|
|
114
|
+
:instance_type => if!(:vpc_enabled, map!(:config, 'Flavor', :vpc), map!(:config, 'Flavor', :classic)),
|
|
115
|
+
:service_token => ref!(:jackal_sns_topic)
|
|
116
|
+
) do
|
|
117
|
+
properties do
|
|
118
|
+
security_groups [ref!(:jackal_ec2_security_group)]
|
|
119
|
+
network_interfaces if!(
|
|
120
|
+
:vpc_enabled,
|
|
121
|
+
array!(
|
|
122
|
+
-> {
|
|
123
|
+
device_index 0
|
|
124
|
+
associate_public_ip_address 'true'
|
|
125
|
+
subnet_id select!(0, ref!(:networking_subnet_ids))
|
|
126
|
+
}
|
|
127
|
+
),
|
|
128
|
+
[]
|
|
129
|
+
)
|
|
130
|
+
end
|
|
131
|
+
metadata('AWS::CloudFormation::Init') do
|
|
132
|
+
camel_keys_set!(:auto_disable)
|
|
133
|
+
config do
|
|
134
|
+
files('/etc/jackal/configuration.json') do
|
|
135
|
+
content do
|
|
136
|
+
jackal do
|
|
137
|
+
require [
|
|
138
|
+
'carnivore-sqs',
|
|
139
|
+
'jackal-cfn'
|
|
140
|
+
]
|
|
141
|
+
cfn do
|
|
142
|
+
sources do
|
|
143
|
+
input do
|
|
144
|
+
type :sqs
|
|
145
|
+
args do
|
|
146
|
+
fog do
|
|
147
|
+
aws_access_key_id ref!(:jackal_iam_access_key)
|
|
148
|
+
aws_secret_access_key attr!(:jackal_iam_access_key, :secret_access_key)
|
|
149
|
+
region region!
|
|
150
|
+
end
|
|
151
|
+
queues [
|
|
152
|
+
ref!(:jackal_sqs_queue)
|
|
153
|
+
]
|
|
154
|
+
end
|
|
155
|
+
end
|
|
156
|
+
end
|
|
157
|
+
config do
|
|
158
|
+
reprocess true
|
|
159
|
+
ami.credentials.compute do
|
|
160
|
+
aws_access_key_id ref!(:jackal_iam_access_key)
|
|
161
|
+
aws_secret_access_key attr!(:jackal_iam_access_key, :secret_access_key)
|
|
162
|
+
region region!
|
|
163
|
+
end
|
|
164
|
+
jackal_stack do
|
|
165
|
+
credentials do
|
|
166
|
+
storage do
|
|
167
|
+
aws_access_key_id ref!(:jackal_iam_access_key)
|
|
168
|
+
aws_secret_access_key attr!(:jackal_iam_access_key, :secret_access_key)
|
|
169
|
+
end
|
|
170
|
+
us_west_1 do
|
|
171
|
+
provider "aws"
|
|
172
|
+
aws_access_key_id ref!(:jackal_iam_access_key)
|
|
173
|
+
aws_secret_access_key attr!(:jackal_iam_access_key, :secret_access_key)
|
|
174
|
+
aws_region 'us-west-1'
|
|
175
|
+
end
|
|
176
|
+
end
|
|
177
|
+
end
|
|
178
|
+
end
|
|
179
|
+
callbacks [
|
|
180
|
+
'Jackal::Cfn::Resource',
|
|
181
|
+
'Jackal::Cfn::AmiRegister',
|
|
182
|
+
'Jackal::Cfn::AmiManager',
|
|
183
|
+
'Jackal::Cfn::JackalStack',
|
|
184
|
+
'Jackal::Cfn::OrchestrationUnit'
|
|
185
|
+
]
|
|
186
|
+
end
|
|
187
|
+
end
|
|
188
|
+
end
|
|
189
|
+
end
|
|
190
|
+
files('/etc/init.d/jackal') do
|
|
191
|
+
content "#!/bin/sh\nstart-stop-daemon --start --oknodo --user jackal --pidfile /var/run/jackal.pid --make-pidfile --chuid jackal --background --exec /bin/bash -- -c 'jackal -c /etc/jackal > /opt/jackal/run.log 2>&1'\n"
|
|
192
|
+
mode '000755'
|
|
193
|
+
end
|
|
194
|
+
files('/jackal-example.json') do
|
|
195
|
+
content.set!('OriginStack', stack_id!)
|
|
196
|
+
end
|
|
197
|
+
users.jackal.homeDir '/opt/jackal'
|
|
198
|
+
commands('00_ntp_sync') do
|
|
199
|
+
command 'apt-get install ntpdate -qy && ntpdate -b -s pool.ntp.org'
|
|
200
|
+
end
|
|
201
|
+
commands('01_jackal_directory') do
|
|
202
|
+
command 'mkdir /opt/jackal && chown jackal /opt/jackal'
|
|
203
|
+
end
|
|
204
|
+
commands('02_custom_packages') do
|
|
205
|
+
command join!(
|
|
206
|
+
'apt-get install -yq ',
|
|
207
|
+
join!(
|
|
208
|
+
ref!(:jackal_custom_packages),
|
|
209
|
+
:options => {
|
|
210
|
+
:delimiter => ' '
|
|
211
|
+
}
|
|
212
|
+
)
|
|
213
|
+
)
|
|
214
|
+
ignoreErrors 'true'
|
|
215
|
+
end
|
|
216
|
+
commands('02_software_properties_old') do
|
|
217
|
+
command 'apt-get install -yq python-software-properties'
|
|
218
|
+
ignoreErrors 'true'
|
|
219
|
+
end
|
|
220
|
+
commands('02_software_properties_new') do
|
|
221
|
+
command 'apt-get install -yq software-properties-common'
|
|
222
|
+
ignoreErrors 'true'
|
|
223
|
+
end
|
|
224
|
+
commands('03_add_repository') do
|
|
225
|
+
command 'apt-add-repository ppa:brightbox/ruby-ng -y'
|
|
226
|
+
end
|
|
227
|
+
commands('04_apt_reupdate') do
|
|
228
|
+
command 'apt-get update'
|
|
229
|
+
end
|
|
230
|
+
commands('05_ruby_install') do
|
|
231
|
+
command 'apt-get install -qy ruby2.2 ruby2.2-dev libyajl-dev build-essential libcurl3-dev libxslt1-dev libxml2 zlib1g-dev awscli'
|
|
232
|
+
end
|
|
233
|
+
commands('06_jackal_install') do
|
|
234
|
+
command 'gem install --no-document jackal-cfn carnivore-sqs'
|
|
235
|
+
end
|
|
236
|
+
commands('07_jackal_startup') do
|
|
237
|
+
command '/etc/init.d/jackal start'
|
|
238
|
+
end
|
|
239
|
+
commands('08_pause_for_file_stabilization') do
|
|
240
|
+
command 'sleep 180'
|
|
241
|
+
end
|
|
242
|
+
commands('09_notify_complete') do
|
|
243
|
+
command join!(
|
|
244
|
+
"cfn-signal -e 0 -r 'Provision complete' --resource #{resource_name!} --region ",
|
|
245
|
+
region!,
|
|
246
|
+
' --stack ',
|
|
247
|
+
stack_name!
|
|
248
|
+
)
|
|
249
|
+
end
|
|
250
|
+
commands('10_pause_for_dramatic_effect') do
|
|
251
|
+
command 'sleep 3500'
|
|
252
|
+
end
|
|
253
|
+
commands('11_kill_self') do
|
|
254
|
+
command join!(
|
|
255
|
+
'aws ec2 terminate-instances --instance-ids `curl -s http://169.254.169.254/latest/meta-data/instance-id` --region ',
|
|
256
|
+
region!
|
|
257
|
+
)
|
|
258
|
+
env do
|
|
259
|
+
set!('AWS_ACCESS_KEY_ID', ref!(:jackal_iam_access_key))
|
|
260
|
+
set!('AWS_SECRET_ACCESS_KEY', attr!(:jackal_iam_access_key, :secret_access_key))
|
|
261
|
+
end
|
|
262
|
+
end
|
|
263
|
+
end
|
|
264
|
+
end
|
|
265
|
+
end
|
|
266
|
+
|
|
267
|
+
dynamic!(:auto_scaling_launch_configuration, :jackal) do
|
|
268
|
+
properties do
|
|
269
|
+
image_id attr!(:jackal_cfn_jackal_image, :ami_id)
|
|
270
|
+
key_name ref!(:jackal_cfn_key_name)
|
|
271
|
+
instance_type if!(:vpc_enabled, map!(:config, 'Flavor', :vpc), map!(:config, 'Flavor', :classic))
|
|
272
|
+
security_groups [ref!(:jackal_ec2_security_group)]
|
|
273
|
+
user_data base64!(
|
|
274
|
+
join!(
|
|
275
|
+
"#!/bin/bash\n",
|
|
276
|
+
'/usr/local/bin/cfn-init -v --region ',
|
|
277
|
+
region!,
|
|
278
|
+
' -s ',
|
|
279
|
+
stack_name!,
|
|
280
|
+
" -r #{resource_name!}"
|
|
281
|
+
)
|
|
282
|
+
)
|
|
283
|
+
end
|
|
284
|
+
metadata('AWS::CloudFormation::Init') do
|
|
285
|
+
camel_keys_set!(:auto_disable)
|
|
286
|
+
config do
|
|
287
|
+
files('/usr/local/bin/suicider') do
|
|
288
|
+
content join!(
|
|
289
|
+
"#!/bin/bash\n",
|
|
290
|
+
"while true\ndo\nsleep 3500\n",
|
|
291
|
+
"tail -n 1 /opt/jackal/run.log | grep 'no message received'\n",
|
|
292
|
+
"if [ $? -eq 0 ]\nthen\n",
|
|
293
|
+
'AWS_ACCESS_KEY_ID="',
|
|
294
|
+
ref!(:jackal_iam_access_key),
|
|
295
|
+
'" AWS_SECRET_ACCESS_KEY="',
|
|
296
|
+
attr!(:jackal_iam_access_key, :secret_access_key),
|
|
297
|
+
'" aws autoscaling terminate-instance-in-autoscaling-group --should-decrement-desired-capacity --instance-id `curl -s http://169.254.169.254/latest/meta-data/instance-id` --region ',
|
|
298
|
+
region!,
|
|
299
|
+
"\nexit 0\nfi\ndone\n"
|
|
300
|
+
)
|
|
301
|
+
mode '000700'
|
|
302
|
+
end
|
|
303
|
+
commands('00_jackal_init') do
|
|
304
|
+
command '/etc/init.d/jackal start'
|
|
305
|
+
end
|
|
306
|
+
commands('01_suicider') do
|
|
307
|
+
command 'nohup bash -c /usr/local/bin/suicider &'
|
|
308
|
+
end
|
|
309
|
+
end
|
|
310
|
+
end
|
|
311
|
+
end
|
|
312
|
+
|
|
313
|
+
dynamic!(:auto_scaling_group, :jackal) do
|
|
314
|
+
properties do
|
|
315
|
+
cooldown 500
|
|
316
|
+
max_size ref!(:jackal_max_cluster_size)
|
|
317
|
+
min_size 0
|
|
318
|
+
availability_zones if!(:vpc_enabled, no_value!, azs!)
|
|
319
|
+
VPCZoneIdentifier if!(:vpc_enabled, ref!(:networking_subnet_ids), no_value!)
|
|
320
|
+
launch_configuration_name ref!(:jackal_auto_scaling_launch_configuration)
|
|
321
|
+
end
|
|
322
|
+
end
|
|
323
|
+
|
|
324
|
+
dynamic!(:auto_scaling_scaling_policy, :jackal) do
|
|
325
|
+
properties do
|
|
326
|
+
adjustment_type 'ChangeInCapacity'
|
|
327
|
+
auto_scaling_group_name ref!(:jackal_auto_scaling_group)
|
|
328
|
+
cooldown 600
|
|
329
|
+
scaling_adjustment 1
|
|
330
|
+
end
|
|
331
|
+
end
|
|
332
|
+
|
|
333
|
+
dynamic!(:cloud_watch_alarm, :jackal) do
|
|
334
|
+
properties do
|
|
335
|
+
actions_enabled true
|
|
336
|
+
alarm_actions [ref!(:jackal_auto_scaling_scaling_policy)]
|
|
337
|
+
alarm_description 'Jackal Kickstart'
|
|
338
|
+
comparison_operator 'GreaterThanOrEqualToThreshold'
|
|
339
|
+
evaluation_periods 2
|
|
340
|
+
metric_name 'ApproximateNumberOfMessagesVisible'
|
|
341
|
+
namespace 'AWS/SQS'
|
|
342
|
+
dimensions array!(
|
|
343
|
+
->{
|
|
344
|
+
name 'QueueName'
|
|
345
|
+
value attr!(:jackal_sqs_queue, :queue_name)
|
|
346
|
+
}
|
|
347
|
+
)
|
|
348
|
+
period 60
|
|
349
|
+
statistic 'Average'
|
|
350
|
+
threshold 1
|
|
351
|
+
end
|
|
352
|
+
end
|
|
353
|
+
|
|
354
|
+
resources.jackal_image_cleanup do
|
|
355
|
+
type 'Custom::AmiManager'
|
|
356
|
+
depends_on!(
|
|
357
|
+
:jackal_cloud_watch_alarm,
|
|
358
|
+
:jackal_sqs_queue_policy,
|
|
359
|
+
:jackal_auto_scaling_scaling_policy,
|
|
360
|
+
:jackal_auto_scaling_group
|
|
361
|
+
)
|
|
362
|
+
properties do
|
|
363
|
+
service_token ref!(:jackal_sns_topic)
|
|
364
|
+
parameters do
|
|
365
|
+
ami_id attr!(:jackal_cfn_jackal_image, :ami_id)
|
|
366
|
+
region region!
|
|
367
|
+
end
|
|
368
|
+
end
|
|
369
|
+
end
|
|
370
|
+
|
|
371
|
+
end
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
SparkleFormation.new(:jackal_cfn_usage_example) do
|
|
2
|
+
|
|
3
|
+
AWSTemplateFormatVersion '2010-09-09'
|
|
4
|
+
description 'Jackal CFN Example Usage'
|
|
5
|
+
|
|
6
|
+
parameters.jackal_service_token.type 'String'
|
|
7
|
+
|
|
8
|
+
dynamic!(:orchestration_unit, :example).properties.parameters.exec 'cat /jackal-example.json'
|
|
9
|
+
|
|
10
|
+
outputs do
|
|
11
|
+
full_result.value attr!(:example_orchestration_unit, :orchestration_unit_result)
|
|
12
|
+
key_result.value attr!(:example_orchestration_unit, :origin_stack)
|
|
13
|
+
end
|
|
14
|
+
|
|
15
|
+
end
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
Gem::Specification.new do |s|
|
|
2
|
+
s.name = 'sparkle-pack-jackal-cfn'
|
|
3
|
+
s.version = '0.1.0'
|
|
4
|
+
s.summary = 'Jackal CFN compute stacker'
|
|
5
|
+
s.author = 'Chris Roberts'
|
|
6
|
+
s.email = 'chrisroberts.code@gmail.com'
|
|
7
|
+
s.homepage = 'http://github.com/carnivore-rb/sparkle-pack-jackal-cfn'
|
|
8
|
+
s.description = 'SparkleFormation pack for generating Jackal CFN compute stack'
|
|
9
|
+
s.license = 'Apache-2.0'
|
|
10
|
+
s.require_path = 'lib'
|
|
11
|
+
s.add_runtime_dependency 'sparkle_formation', '>= 2.1.0'
|
|
12
|
+
s.files = Dir['{lib,docs}/**/*'] + %w(sparkle-pack-jackal-cfn.gemspec README.md CHANGELOG.md LICENSE)
|
|
13
|
+
end
|
metadata
ADDED
|
@@ -0,0 +1,72 @@
|
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
|
2
|
+
name: sparkle-pack-jackal-cfn
|
|
3
|
+
version: !ruby/object:Gem::Version
|
|
4
|
+
version: 0.1.0
|
|
5
|
+
platform: ruby
|
|
6
|
+
authors:
|
|
7
|
+
- Chris Roberts
|
|
8
|
+
autorequire:
|
|
9
|
+
bindir: bin
|
|
10
|
+
cert_chain: []
|
|
11
|
+
date: 2016-02-08 00:00:00.000000000 Z
|
|
12
|
+
dependencies:
|
|
13
|
+
- !ruby/object:Gem::Dependency
|
|
14
|
+
name: sparkle_formation
|
|
15
|
+
requirement: !ruby/object:Gem::Requirement
|
|
16
|
+
requirements:
|
|
17
|
+
- - ">="
|
|
18
|
+
- !ruby/object:Gem::Version
|
|
19
|
+
version: 2.1.0
|
|
20
|
+
type: :runtime
|
|
21
|
+
prerelease: false
|
|
22
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
23
|
+
requirements:
|
|
24
|
+
- - ">="
|
|
25
|
+
- !ruby/object:Gem::Version
|
|
26
|
+
version: 2.1.0
|
|
27
|
+
description: SparkleFormation pack for generating Jackal CFN compute stack
|
|
28
|
+
email: chrisroberts.code@gmail.com
|
|
29
|
+
executables: []
|
|
30
|
+
extensions: []
|
|
31
|
+
extra_rdoc_files: []
|
|
32
|
+
files:
|
|
33
|
+
- CHANGELOG.md
|
|
34
|
+
- LICENSE
|
|
35
|
+
- README.md
|
|
36
|
+
- lib/sparkle-pack-jackal-cfn.rb
|
|
37
|
+
- lib/sparkleformation/dynamics/ami_manager.rb
|
|
38
|
+
- lib/sparkleformation/dynamics/ami_register.rb
|
|
39
|
+
- lib/sparkleformation/dynamics/hash_extractor.rb
|
|
40
|
+
- lib/sparkleformation/dynamics/jackal_image.rb
|
|
41
|
+
- lib/sparkleformation/dynamics/jackal_stack.rb
|
|
42
|
+
- lib/sparkleformation/dynamics/orchestration_unit.rb
|
|
43
|
+
- lib/sparkleformation/jackal_bus.rb
|
|
44
|
+
- lib/sparkleformation/jackal_cfn.rb
|
|
45
|
+
- lib/sparkleformation/jackal_usage_example.rb
|
|
46
|
+
- sparkle-pack-jackal-cfn.gemspec
|
|
47
|
+
homepage: http://github.com/carnivore-rb/sparkle-pack-jackal-cfn
|
|
48
|
+
licenses:
|
|
49
|
+
- Apache-2.0
|
|
50
|
+
metadata: {}
|
|
51
|
+
post_install_message:
|
|
52
|
+
rdoc_options: []
|
|
53
|
+
require_paths:
|
|
54
|
+
- lib
|
|
55
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
|
56
|
+
requirements:
|
|
57
|
+
- - ">="
|
|
58
|
+
- !ruby/object:Gem::Version
|
|
59
|
+
version: '0'
|
|
60
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
61
|
+
requirements:
|
|
62
|
+
- - ">="
|
|
63
|
+
- !ruby/object:Gem::Version
|
|
64
|
+
version: '0'
|
|
65
|
+
requirements: []
|
|
66
|
+
rubyforge_project:
|
|
67
|
+
rubygems_version: 2.4.8
|
|
68
|
+
signing_key:
|
|
69
|
+
specification_version: 4
|
|
70
|
+
summary: Jackal CFN compute stacker
|
|
71
|
+
test_files: []
|
|
72
|
+
has_rdoc:
|