aws-ec2 0.5.1 → 0.5.2
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +3 -0
- data/Gemfile.lock +1 -1
- data/README.md +91 -16
- data/lib/aws-ec2.rb +1 -1
- data/lib/aws_ec2/cli.rb +4 -4
- data/lib/aws_ec2/{compile_scripts.rb → compile.rb} +1 -1
- data/lib/aws_ec2/create/params.rb +0 -3
- data/lib/aws_ec2/help/compile.md +5 -0
- data/lib/aws_ec2/template_helper/ami_helper.rb +6 -2
- data/lib/aws_ec2/version.rb +1 -1
- metadata +3 -6
- data/example/profiles/default.yml +0 -12
- data/example/profiles/spot.yml +0 -9
- data/example/profiles/user-data/dev.sh +0 -47
- data/lib/aws_ec2/help/user_data.md +0 -13
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 0d2b33737beb7cc7f0b0f62447883b3cb7ad5235a4b534359f2128b42cde928b
|
4
|
+
data.tar.gz: b49727f0247691d3f21370fad4ba618064d3c7d69031500241e322edf33ce4bf
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: fe62f5a533ba85f766c956e500c3ca5b7771df36936948c1f89c44f359fec08218343dcbbedef841f9a005f3f7127907cda62d3d8b1c9635232985c4741309db
|
7
|
+
data.tar.gz: e0c4b8db0bf9195693e3be9321376ffa752740d4ee6ac6584b2f44ccbd95f2162e4b489546de2a7672b06b3424f8038dd13d630414d2a2f87255c48b6e327aa7
|
data/CHANGELOG.md
CHANGED
@@ -3,6 +3,9 @@
|
|
3
3
|
All notable changes to this project will be documented in this file.
|
4
4
|
This project *tries* to adhere to [Semantic Versioning](http://semver.org/), even before v1.0.
|
5
5
|
|
6
|
+
## [0.5.2]
|
7
|
+
- remove byebug debugging
|
8
|
+
|
6
9
|
## [0.5.1]
|
7
10
|
- show a friendly user message if not an aws-ec2 project
|
8
11
|
|
data/Gemfile.lock
CHANGED
data/README.md
CHANGED
@@ -3,7 +3,7 @@
|
|
3
3
|
Simple tool to create AWS ec2 instances consistently with pre-configured settings. The pre-configured settings are stored in the profiles folder of the current directory.
|
4
4
|
Example:
|
5
5
|
|
6
|
-
* profiles/default.yml: default settings.
|
6
|
+
* profiles/default.yml: default settings.
|
7
7
|
* profiles/myserver.yml: myserver settings.
|
8
8
|
|
9
9
|
## Usage
|
@@ -14,12 +14,21 @@ aws-ec2 create myserver --profile myserver
|
|
14
14
|
|
15
15
|
In a nutshell, the profile parameters are passed to the ruby aws-sdk [AWS::EC2::Client#run_instances](https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/EC2/Client.html#run_instances-instance_method) method. So you can specify any parameter you wish that is available there. To check out what a profile looks like look at the [example default](example/profiles/default.yml)
|
16
16
|
|
17
|
-
You can use ERB in the profile files. Some useful helper methods in the profile files are:
|
17
|
+
You can use ERB in the profile files. Some useful helper methods in the profile files are documented here:
|
18
18
|
|
19
|
-
|
20
|
-
|
19
|
+
Helper | Description
|
20
|
+
------------- | -------------
|
21
|
+
user_data | Allows you to embed a generated user_data script. More details on the are provided in the user data section below.
|
22
|
+
config | Access to the variables set in config/[AWS_EC2_ENV].yml.
|
23
|
+
latest_ami | Returns an AMI id by searching the ami name pattern and sorting in reverse older. Example: `latest_ami("ruby-2.5.0_*")`
|
24
|
+
search_ami | Returns a collection of AMI image objects based on a search pattern. The query searches on the AMI name.
|
21
25
|
|
22
|
-
The template helpers defined in
|
26
|
+
The template helpers defined in:
|
27
|
+
|
28
|
+
* [aws_ec2/template_helper.rb](lib/aws_ec2/template_helper.rb).
|
29
|
+
* [aws_ec2/template_helper](lib/aws_ec2/template_helper).
|
30
|
+
|
31
|
+
You can also define your own custom helpers in the `app/helpers` folder as ruby modules with the naming convention `'*_helper.rb`. Example, the module FooHelper should be defined in `app/helpers/foo_helper.rb`. The custom helpers are first class citizens and have access to the same variables and methods as built in helpers.
|
23
32
|
|
24
33
|
### Convention
|
25
34
|
|
@@ -29,29 +38,90 @@ By convention, the profile name matches the first parameter after the create com
|
|
29
38
|
aws-ec2 create myserver
|
30
39
|
```
|
31
40
|
|
41
|
+
## Noop mode
|
42
|
+
|
43
|
+
You can do a test run with the `--noop` flag. This will print out what settings will be used to launch the instance. This is a good way to inspect the generated user-data script.
|
44
|
+
|
45
|
+
```sh
|
46
|
+
aws-ec2 create myserver --profile myserver --noop
|
47
|
+
```
|
48
|
+
|
49
|
+
## Project Structure
|
50
|
+
|
51
|
+
Directory | Description
|
52
|
+
------------- | -------------
|
53
|
+
app/helpers | Your own custom helpers methods. Define them in modules and the methods made available to your `config`, `profiles`, `app/scripts`, `app/user-data` files.
|
54
|
+
app/partials | Your partials that can be use to be included in other scripts. This is used in conjunction with the partial helper method.
|
55
|
+
app/scripts | Where you define common scripts that can be used to configure the server.
|
56
|
+
app/user-data | Your user-data scripts that are used to bootstrap EC2 instance.
|
57
|
+
config/[AWS_EC2_ENV].yml | AWS_EC2_ENV can be development, staging or production. Use this config file to set configs that you want available in your templating logic.
|
58
|
+
profiles | Your profile files. These files mainly contain parameters that are passed to the aws-sd2 run_instances API.
|
59
|
+
tmp | Where the generated scripts get compiled to. You can manually invoke the complilation via `aws-ec2 compile`.
|
60
|
+
|
32
61
|
## User-Data
|
33
62
|
|
34
|
-
You can provide user-data script to customize the server upon launch. The user-data scripts are under the app/user-data folder.
|
63
|
+
You can provide user-data script to customize the server upon launch. The user-data scripts are under the `app/user-data` folder.
|
35
64
|
|
36
65
|
* app/user-data/myserver.yml
|
37
66
|
|
38
|
-
The user-data script is generated on the machine that is running the aws-ec2 command. If this is your local macosx machine, then the context is your local macosx machine is available. To see the generated user-data script, you can use the
|
67
|
+
The user-data script is generated on the machine that is running the aws-ec2 command. If this is your local macosx machine, then the context is your local macosx machine is available. To see the generated user-data script, you can use the run the create command in noop mode and then inspect the generated script. Example:
|
68
|
+
|
69
|
+
```sh
|
70
|
+
aws create myserver --noop
|
71
|
+
cat /tmp/aws-ec2/user-data.txt
|
72
|
+
```
|
39
73
|
|
40
|
-
|
74
|
+
To use the user-data script when creating an EC2 instance, you can use the helper method in the profile. Here's a grep of a profile that users the helper:
|
41
75
|
|
42
|
-
|
76
|
+
```
|
77
|
+
$ grep user_data profiles/default.yml
|
78
|
+
user_data: "<%= user_data("bootstrap") %>"
|
79
|
+
```
|
43
80
|
|
44
81
|
### Config
|
45
82
|
|
46
|
-
You can set a config file and define variables in there that are available to in your profiles and user_data scripts.
|
83
|
+
You can set a config file and define variables in there that are available to in your profiles and user_data scripts. Example `config/development.yml`:
|
84
|
+
|
85
|
+
```yaml
|
86
|
+
---
|
87
|
+
vpc_id: vpc-123
|
88
|
+
db_subnet_group_name: default
|
89
|
+
- subnet-123
|
90
|
+
- subnet-456
|
91
|
+
- subnet-789
|
92
|
+
security_group_ids:
|
93
|
+
- sg-123
|
94
|
+
s3_bucket: mybucket # for the user-data shared scripts
|
95
|
+
```
|
47
96
|
|
48
|
-
|
97
|
+
The variables are accessible via the `config` helper method. Example (only showing the part of the profile), `profiles/default.yml`:
|
49
98
|
|
50
|
-
|
99
|
+
```yaml
|
100
|
+
image_id: ami-4fffc834 # Amazon Lambda AMI
|
101
|
+
instance_type: t2.medium
|
102
|
+
security_group_ids: <%= config["security_group_ids"] %>
|
103
|
+
subnet_id: <%= config["subnets"].shuffle %>
|
104
|
+
...
|
105
|
+
```
|
106
|
+
|
107
|
+
## Dotenv File Support
|
108
|
+
|
109
|
+
You can set and configure environment variables in `.env*` files. Examples of this is in the [doc/example](doc/example) project.
|
110
|
+
|
111
|
+
### Hooks
|
112
|
+
|
113
|
+
There is only one hook: before_run_instances. You can configure this with `config/hooks.yml`: Example:
|
51
114
|
|
52
|
-
```sh
|
53
|
-
aws-ec2 create myserver --profile myserver --noop
|
54
115
|
```
|
116
|
+
---
|
117
|
+
before_run_instances: /path/to/my/script.sh
|
118
|
+
```
|
119
|
+
|
120
|
+
## AMI Creation
|
121
|
+
|
122
|
+
To create AMIs you can use the `aws-ec2 ami` command. This command will run launch an EC2 instance with one of the profiles and create an AMI after the user-data script completes successfully. It does this by adding an AMI creation script at the end of the user-data script. It is recommended to use the `set -e` option so that any error halts the script and the AMI does not get created.
|
123
|
+
|
124
|
+
After the AMI is created successfully, the instance will also terminate itself automatically so you do not have to worry about cleanup. For more help run `aws-ec2 ami help`.
|
55
125
|
|
56
126
|
## Spot Instance Support
|
57
127
|
|
@@ -64,11 +134,12 @@ Spot instance support natively supported by the AWS run_instances command. Simp
|
|
64
134
|
|
65
135
|
```sh
|
66
136
|
aws-ec2 create help
|
67
|
-
aws-ec2
|
137
|
+
aws-ec2 ami help
|
138
|
+
aws-ec2 compile help
|
68
139
|
aws-ec2 help # general help
|
69
140
|
```
|
70
141
|
|
71
|
-
Examples are in the [example](example) folder. You will have to update settings like your subnet and security group ids.
|
142
|
+
Examples are in the [doc/example](doc/example) folder. You will have to update settings like your subnet and security group ids.
|
72
143
|
|
73
144
|
## Installation
|
74
145
|
|
@@ -76,6 +147,10 @@ Examples are in the [example](example) folder. You will have to update settings
|
|
76
147
|
gem install aws-ec2
|
77
148
|
```
|
78
149
|
|
150
|
+
### Dependencies
|
151
|
+
|
152
|
+
This tool mainly uses the ruby aws-sdk but it does use the aws cli to check your region: `aws configure get region`. So it is dependent on the the `aws cli`.
|
153
|
+
|
79
154
|
## Contributing
|
80
155
|
|
81
156
|
1. Fork it
|
data/lib/aws-ec2.rb
CHANGED
data/lib/aws_ec2/cli.rb
CHANGED
@@ -20,10 +20,10 @@ module AwsEc2
|
|
20
20
|
Ami.new(options.merge(name: name)).run
|
21
21
|
end
|
22
22
|
|
23
|
-
desc "
|
24
|
-
long_desc Help.text(:
|
25
|
-
def
|
26
|
-
|
23
|
+
desc "compile", "compiles app/scripts and app/user-data to tmp folder"
|
24
|
+
long_desc Help.text(:compile)
|
25
|
+
def compile
|
26
|
+
Compile.new(options).compile
|
27
27
|
end
|
28
28
|
end
|
29
29
|
end
|
@@ -11,13 +11,17 @@ module AwsEc2::TemplateHelper::AmiHelper
|
|
11
11
|
#
|
12
12
|
# Returns latest ami ami
|
13
13
|
def latest_ami(query, owners=["self"])
|
14
|
+
images = search_ami(query, owners)
|
15
|
+
image = images.sort_by(&:name).reverse.first
|
16
|
+
image.image_id
|
17
|
+
end
|
18
|
+
|
19
|
+
def search_ami(query, owners=["self"])
|
14
20
|
images = ec2.describe_images(
|
15
21
|
owners: owners,
|
16
22
|
filters: [
|
17
23
|
{name: "name", values: [query]}
|
18
24
|
]
|
19
25
|
).images
|
20
|
-
image = images.sort_by(&:name).reverse.first
|
21
|
-
image.image_id
|
22
26
|
end
|
23
27
|
end
|
data/lib/aws_ec2/version.rb
CHANGED
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: aws-ec2
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.5.
|
4
|
+
version: 0.5.2
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Tung Nguyen
|
@@ -197,9 +197,6 @@ files:
|
|
197
197
|
- README.md
|
198
198
|
- Rakefile
|
199
199
|
- aws-ec2.gemspec
|
200
|
-
- example/profiles/default.yml
|
201
|
-
- example/profiles/spot.yml
|
202
|
-
- example/profiles/user-data/dev.sh
|
203
200
|
- exe/aws-ec2
|
204
201
|
- lib/aws-ec2.rb
|
205
202
|
- lib/aws_ec2/ami.rb
|
@@ -207,7 +204,7 @@ files:
|
|
207
204
|
- lib/aws_ec2/base.rb
|
208
205
|
- lib/aws_ec2/cli.rb
|
209
206
|
- lib/aws_ec2/command.rb
|
210
|
-
- lib/aws_ec2/
|
207
|
+
- lib/aws_ec2/compile.rb
|
211
208
|
- lib/aws_ec2/config.rb
|
212
209
|
- lib/aws_ec2/core.rb
|
213
210
|
- lib/aws_ec2/create.rb
|
@@ -215,8 +212,8 @@ files:
|
|
215
212
|
- lib/aws_ec2/dotenv.rb
|
216
213
|
- lib/aws_ec2/help.rb
|
217
214
|
- lib/aws_ec2/help/ami.md
|
215
|
+
- lib/aws_ec2/help/compile.md
|
218
216
|
- lib/aws_ec2/help/create.md
|
219
|
-
- lib/aws_ec2/help/user_data.md
|
220
217
|
- lib/aws_ec2/hook.rb
|
221
218
|
- lib/aws_ec2/script.rb
|
222
219
|
- lib/aws_ec2/scripts/ami_creation.sh
|
@@ -1,12 +0,0 @@
|
|
1
|
-
---
|
2
|
-
image_id: ami-97785bed
|
3
|
-
instance_type: t2.medium
|
4
|
-
key_name: default
|
5
|
-
max_count: 1
|
6
|
-
min_count: 1
|
7
|
-
security_group_ids:
|
8
|
-
- sg-111
|
9
|
-
subnet_id: <%= %w[subnet-111 subnet-222].shuffle.first %>
|
10
|
-
user_data: "<%= user_data("dev") %>"
|
11
|
-
iam_instance_profile:
|
12
|
-
name: IAMProfileName
|
data/example/profiles/spot.yml
DELETED
@@ -1,9 +0,0 @@
|
|
1
|
-
---
|
2
|
-
instance_market_options:
|
3
|
-
market_type: spot
|
4
|
-
# https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_LaunchTemplateSpotMarketOptionsRequest.html
|
5
|
-
# https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/EC2/Types/SpotMarketOptions.html
|
6
|
-
spot_options:
|
7
|
-
max_price: "0.02"
|
8
|
-
spot_instance_type: one-time
|
9
|
-
# instance_interruption_behavior: hibernate
|
@@ -1,47 +0,0 @@
|
|
1
|
-
#!/bin/bash -exu
|
2
|
-
|
3
|
-
exec > >(tee "/var/log/user-data.log" | logger -t user-data -s 2>/dev/console) 2>&1
|
4
|
-
export HOME=/root # user-data env runs in weird shell where user is root but HOME is not set
|
5
|
-
<% pubkey_path = "#{ENV['HOME']}/.ssh/id_rsa.pub" -%>
|
6
|
-
<% if File.exist?(pubkey_path) -%>
|
7
|
-
<% pubkey = IO.read(pubkey_path).strip -%>
|
8
|
-
# Automatically add user's public key
|
9
|
-
echo <%= pubkey %> >> ~/.ssh/authorized_keys
|
10
|
-
echo <%= pubkey %> >> /home/ec2-user/.ssh/authorized_keys
|
11
|
-
chown ec2-user:ec2-user /home/ec2-user/.ssh/authorized_keys
|
12
|
-
<% else %>
|
13
|
-
# WARN: unable to find a ~/.ssh/id_rsa.pub locally on your machine. user: <%= ENV['USER'] %>
|
14
|
-
# Unable to automatically add the public key
|
15
|
-
<% end -%>
|
16
|
-
|
17
|
-
sudo yum install -y postgresql
|
18
|
-
|
19
|
-
# https://gist.github.com/juno/1330165
|
20
|
-
# Install developer tools
|
21
|
-
yum install -y git gcc make readline-devel openssl-devel
|
22
|
-
|
23
|
-
# Install ruby-build system-widely
|
24
|
-
git clone git://github.com/sstephenson/ruby-build.git /tmp/ruby-build
|
25
|
-
cd /tmp/ruby-build
|
26
|
-
./install.sh
|
27
|
-
echo 'export PATH="/usr/local/bin:$PATH"' >> ~/.bashrc
|
28
|
-
|
29
|
-
# Install rbenv for root
|
30
|
-
git clone git://github.com/sstephenson/rbenv.git ~/.rbenv
|
31
|
-
echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc
|
32
|
-
echo 'eval "$(rbenv init -)"' >> ~/.bashrc
|
33
|
-
set +u
|
34
|
-
source ~/.bashrc
|
35
|
-
set -u
|
36
|
-
|
37
|
-
# Install and enable ruby
|
38
|
-
rbenv install 2.5.0
|
39
|
-
|
40
|
-
# Install ruby for ec2-user also
|
41
|
-
cp -R ~/.rbenv /home/ec2-user/
|
42
|
-
chown -R ec2-user:ec2-user /home/ec2-user/.rbenv
|
43
|
-
echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> /home/ec2-user/.bashrc
|
44
|
-
echo 'eval "$(rbenv init -)"' >> /home/ec2-user/.bashrc
|
45
|
-
echo '2.5.0' > /home/ec2-user/.ruby-version
|
46
|
-
|
47
|
-
uptime | tee /var/log/boot-time.log
|
@@ -1,13 +0,0 @@
|
|
1
|
-
Displays the generated user data script. Useful for debugging since ERB can be ran on the user-data scripts.
|
2
|
-
|
3
|
-
Given a user data script in app/user-data/myscript.sh, run:
|
4
|
-
|
5
|
-
$ aws-ec2 userdata myscript
|
6
|
-
|
7
|
-
You can have an ami creation snippet of code added to the end of the user data script with the `--ami` option.
|
8
|
-
|
9
|
-
$ aws-ec2 userdata myscript --ami myname
|
10
|
-
|
11
|
-
If you want to include a timestamp in the name you can use this:
|
12
|
-
|
13
|
-
$ aws-ec2 userdata myscript --ami '`date "+myname_%Y-%m-%d-%H-%M"`'
|