ufo 0.1.6 → 1.0.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +5 -0
- data/Gemfile +0 -1
- data/README.md +86 -70
- data/lib/starter_project/ufo/task_definitions.rb +5 -0
- data/lib/ufo.rb +1 -0
- data/lib/ufo/cli.rb +40 -15
- data/lib/ufo/cli/help.rb +22 -0
- data/lib/ufo/settings.rb +1 -1
- data/lib/ufo/ship.rb +0 -30
- data/lib/ufo/task.rb +53 -0
- data/lib/ufo/version.rb +1 -1
- data/spec/lib/cli_spec.rb +7 -0
- data/spec/lib/ecr_cleaner_spec.rb +7 -1
- data/spec/lib/ship_spec.rb +26 -32
- data/spec/lib/task_spec.rb +33 -0
- data/spec/spec_helper.rb +6 -1
- metadata +6 -3
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: dfca9e6d1cee650f151b4ad478f5fd3fb48ab840
|
4
|
+
data.tar.gz: 76d021b7c76d3215fe4fbddd07785c26d889c31c
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 23b34f792eeff2b250a5215a2b44c99405aa04cc33bc985c1f5ccbc6ee66587b25bc98f869e4cc3f8c03a4710a38db5147457c606cdcafedc9ca700ea420cdee
|
7
|
+
data.tar.gz: 5eb6c95ab9839e658f9cd4dae7275d4d1bf0c6c784f92d169e4c036e85fd341b0a36d6412707dd0e681a7978c2f2553c12d6cd9b1644650896423326b8ee310a
|
data/CHANGELOG.md
CHANGED
@@ -3,6 +3,11 @@
|
|
3
3
|
All notable changes to this project will be documented in this file.
|
4
4
|
This project *tries* to adhere to [Semantic Versioning](http://semver.org/), even before v1.0.
|
5
5
|
|
6
|
+
## [1.0.0]
|
7
|
+
|
8
|
+
* add `ufo task` command
|
9
|
+
* been using in production for a while, ready for 1.0.0 release
|
10
|
+
|
6
11
|
## [0.1.6]
|
7
12
|
|
8
13
|
* default wait for deployment to false
|
data/Gemfile
CHANGED
data/README.md
CHANGED
@@ -1,4 +1,6 @@
|
|
1
|
-
# Ufo -
|
1
|
+
# Ufo - A Easy Way to Build and Ship Docker Images AWS ECS
|
2
|
+
|
3
|
+
[![CircleCI](https://circleci.com/gh/tongueroo/ufo.svg?style=svg)](https://circleci.com/gh/tongueroo/ufo)
|
2
4
|
|
3
5
|
## Quick Introduction
|
4
6
|
|
@@ -7,11 +9,11 @@ Ufo is a simple tool that makes building and shipping Docker containers to [AWS
|
|
7
9
|
* This blog post provides an introduction to the tool: [Ufo - Build Docker Containers and Ship Them to AWS ECS](https://medium.com/@tongueroo/ufo-easily-build-docker-containers-and-ship-them-to-aws-ecs-15556a2b39f#.qqu8o4wal).
|
8
10
|
* This presentation covers ufo also: [Ufo Ship on AWS ECS](http://www.slideshare.net/tongueroo/ufo-ship-for-aws-ecs-70885296)
|
9
11
|
|
10
|
-
A summary of
|
12
|
+
A summary of what `ufo ship` does:
|
11
13
|
|
12
|
-
1.
|
13
|
-
2.
|
14
|
-
3.
|
14
|
+
1. Builds a docker image.
|
15
|
+
2. Generates and registers the ECS template definition.
|
16
|
+
3. Deploys the ECS template definition to the specified service.
|
15
17
|
|
16
18
|
Ufo deploys a task definition that is created via a template generator which is fully controllable.
|
17
19
|
|
@@ -20,13 +22,46 @@ Ufo deploys a task definition that is created via a template generator which is
|
|
20
22
|
|
21
23
|
$ gem install ufo
|
22
24
|
|
23
|
-
You will need a working version of docker installed as ufo calls the docker command.
|
25
|
+
Dependencies: You will need a working version of docker installed as ufo shells out and calls the `docker` command.
|
26
|
+
|
27
|
+
## Quick Usage
|
28
|
+
|
29
|
+
### ufo ship
|
24
30
|
|
25
|
-
|
31
|
+
To execute the ship process run:
|
26
32
|
|
27
|
-
|
33
|
+
```bash
|
34
|
+
ufo ship hi-web-prod--cluster mycluster
|
35
|
+
```
|
28
36
|
|
29
|
-
|
37
|
+
Note, if you have configured `ufo/settings.yml` to map hi-web-prod to the `mycluster` cluster using the service_cluster option the command becomes simply:
|
38
|
+
|
39
|
+
```bash
|
40
|
+
ufo ship hi-web-prod
|
41
|
+
```
|
42
|
+
|
43
|
+
When you run `ufo ship hi-web-prod`:
|
44
|
+
|
45
|
+
1. It builds the docker image.
|
46
|
+
2. Generates a task definition and registers it.
|
47
|
+
3. Updates the ECS service to use it.
|
48
|
+
|
49
|
+
If the ECS service hi-web-prod does not yet exist, ufo will create the service for you.
|
50
|
+
|
51
|
+
By convention, if the service has a container name web, you'll get prompted to create an ELB and specify a target group arn. The ELB and target group must already exist.
|
52
|
+
|
53
|
+
You can bypass the prompt and specify the target group arn as part of the command. The elb target group can only be associated when the service gets created for the first time. If the service already exists then the `--target-group` parameter just gets ignored and the ECS task simply gets updated. Example:
|
54
|
+
|
55
|
+
|
56
|
+
```bash
|
57
|
+
ufo ship hi-web-prod --target-group=arn:aws:elasticloadbalancing:us-east-1:12345689:targetgroup/hi-web-prod/12345
|
58
|
+
```
|
59
|
+
|
60
|
+
When using ufo if the ECS service does not yet exist, it will automatically be created for you. Ufo will also automatically create the ECS cluster. If you are relying on this tool to create the cluster, you still need to associate ECS Container Instances to the cluster yourself.
|
61
|
+
|
62
|
+
## Detailed Usage
|
63
|
+
|
64
|
+
First initialize ufo files within your project. Let's say you have an example `hi` app.
|
30
65
|
|
31
66
|
```
|
32
67
|
$ git clone https://github.com/tongueroo/hi
|
@@ -43,30 +78,30 @@ Starter ufo files created.
|
|
43
78
|
$
|
44
79
|
```
|
45
80
|
|
46
|
-
Take a look at the `ufo/settings.yml` file
|
81
|
+
Take a look at the `ufo/settings.yml` file and notice that it contains some default configuration settings so you do not have to type out these options repeatedly for some of the ufo commands.
|
47
82
|
|
48
83
|
```yaml
|
49
84
|
image: tongueroo/hi
|
50
85
|
service_cluster:
|
51
|
-
default:
|
52
|
-
hi-web:
|
53
|
-
hi-clock:
|
54
|
-
hi-worker:
|
86
|
+
default: prod # default cluster
|
87
|
+
hi-web-prod: blue
|
88
|
+
hi-clock-prod: blue
|
89
|
+
hi-worker-prod: blue
|
55
90
|
```
|
56
91
|
|
57
92
|
The `image` value is the name that ufo will use for the Docker image name.
|
58
93
|
|
59
|
-
The `service_cluster` mapping provides a way to set default service
|
94
|
+
The `service_cluster` mapping provides a way to set default service-to-cluster mappings so that you do not have to specify the `--cluster` repeatedly. This is very helpful. For example:
|
60
95
|
|
61
96
|
```
|
62
|
-
ufo ship hi-web --cluster hi-cluster
|
63
|
-
ufo ship hi-web # same as above because it is configured in ufo/settings.yml
|
64
|
-
ufo ship hi-web --cluster special-cluster # overrides
|
97
|
+
ufo ship hi-web-prod --cluster hi-cluster
|
98
|
+
ufo ship hi-web-prod # same as above because it is configured in ufo/settings.yml
|
99
|
+
ufo ship hi-web-prod --cluster special-cluster # overrides the default setting in `ufo/settings.yml`.
|
65
100
|
```
|
66
101
|
|
67
|
-
|
102
|
+
### Task Definition ERB Template and DSL Generator
|
68
103
|
|
69
|
-
Ufo task definitions are
|
104
|
+
Ufo task definitions are written as an ERB template that makes it every easily accessible and configurable to your requirements. Here is is an example of an ERB template `ufo/templates/main.json.erb` that shows how easy it is to modfied the task definition you want to be uploaded by ufo:
|
70
105
|
|
71
106
|
```json
|
72
107
|
{
|
@@ -110,16 +145,16 @@ Ufo task definitions are is written in a template generator DSL to provide full
|
|
110
145
|
}
|
111
146
|
```
|
112
147
|
|
113
|
-
The instance variable values are specified in `ufo/task_definitions.rb
|
148
|
+
The instance variable values are specified in `ufo/task_definitions.rb` via a DSL. Here's the
|
114
149
|
|
115
150
|
|
116
151
|
```ruby
|
117
|
-
task_definition "hi-web" do
|
152
|
+
task_definition "hi-web-prod" do
|
118
153
|
source "main" # will use ufo/templates/main.json.erb
|
119
154
|
variables(
|
120
155
|
family: task_definition_name,
|
121
156
|
# image: tongueroo/hi:ufo-[timestamp]=[sha]
|
122
|
-
image: helper.full_image_name,
|
157
|
+
image: helper.full_image_name,
|
123
158
|
environment: env_file('.env.prod')
|
124
159
|
name: "web",
|
125
160
|
container_port: helper.dockerfile_port,
|
@@ -128,53 +163,15 @@ task_definition "hi-web" do
|
|
128
163
|
end
|
129
164
|
```
|
130
165
|
|
131
|
-
|
166
|
+
The task\_definitions.rb file has some special variables and helper methods available. These helper methods provide useful contextual information about the project. For example, one of the variable provides the exposed port in the Dockerfile of the project. This why if someone changes the exported port in the Dockerfile, he will not "forgot" to also update the ufo variable as it is automatically referenced. Here is a list of the important helpers:
|
132
167
|
|
133
|
-
* **helper.full\_image\_name** — The full docker image name that ufo builds. The “base” portion of the docker image name is defined in ufo/settings.yml. For example, the base portion is
|
168
|
+
* **helper.full\_image\_name** — The full docker image name that ufo builds. The “base” portion of the docker image name is defined in ufo/settings.yml. For example, the base portion is `tongueroo/hi` and the full image name is `tongueroo/hi:ufo-[timestamp]-[sha]`. The base name does not include the generated Docker tag, which contains a timestamp and git sha of the Dockerfile that is used.
|
134
169
|
* **helper.dockerfile\_port** — Exposed port extracted from the Dockerfile of the project.
|
135
|
-
* **
|
170
|
+
* **env_vars(text)** — This method takes a block of text that contains the env values in key=value format and converts that block of text to the proper task definition json format.
|
171
|
+
* **env_file(path)** — This method takes an `.env` file which contains a simple key value list of environment variables and converts the list to the proper task definition json format.
|
136
172
|
|
137
173
|
The 2 classes which provide these special helper methods are in [ufo/dsl.rb](https://github.com/tongueroo/ufo/blob/master/lib/ufo/dsl.rb) and [ufo/dsl/helper.rb](https://github.com/tongueroo/ufo/blob/master/lib/ufo/dsl/helper.rb). Refer to these classes for the full list of the special variables and methods.
|
138
174
|
|
139
|
-
### Customizing Templates
|
140
|
-
|
141
|
-
If you want to change the template then you can follow the example in the generated ufo files. For example, if you want to create a template for the worker service.
|
142
|
-
|
143
|
-
1. Create a new template under ufo/templates/worker.json.erb.
|
144
|
-
2. Change the source in the `task_definition` using "worker" as the source.
|
145
|
-
3. Add variables.
|
146
|
-
|
147
|
-
### ufo ship
|
148
|
-
|
149
|
-
Ufo uses the aforementioned files to build task definitions and then ship to them to AWS ECS. To execute the ship process run:
|
150
|
-
|
151
|
-
```bash
|
152
|
-
ufo ship hi-web --cluster stag
|
153
|
-
```
|
154
|
-
|
155
|
-
Note, if you have configured `ufo/settings.yml` to map hi-web to the stag cluster using the service_cluster option the command becomes simply:
|
156
|
-
|
157
|
-
```bash
|
158
|
-
ufo ship hi-web
|
159
|
-
```
|
160
|
-
|
161
|
-
When you run `ufo ship hi-web`:
|
162
|
-
|
163
|
-
1. It builds the docker image.
|
164
|
-
2. Generates a task definition and registers it.
|
165
|
-
3. Updates the ECS service to use it.
|
166
|
-
|
167
|
-
If the ECS service hi-web does not yet exist, ufo will create the service for you.
|
168
|
-
|
169
|
-
If the service has a container name web, you'll get prompted to create an ELB and specify a target group arn. The ELB and target group must already exist.
|
170
|
-
|
171
|
-
You can bypass the prompt and specify the target group arn as part of the command. The elb target group can only be associated when the service gets created for the first time. If the service already exists then the `--target-group` parameter just gets ignored and the ECS task simply gets updated. Example:
|
172
|
-
|
173
|
-
|
174
|
-
```bash
|
175
|
-
ufo ship hi-web --target-group=arn:aws:elasticloadbalancing:us-east-1:12345689:targetgroup/hi-web/jdisljflsdkjl
|
176
|
-
```
|
177
|
-
|
178
175
|
### Shipping Multiple Services with bin/deploy
|
179
176
|
|
180
177
|
A common pattern is to have 3 processes: web, worker, and clock. This is very common in rails applcations. The web process handles web traffic, the worker process handles background job processing that would be too slow and potentially block web requests, and a clock process is typically used to schedule recurring jobs. These processes use the same codebase, or same docker image, but have slightly different run time settings. For example, the docker run command for a web process could be [puma](http://puma.io/) and the command for a worker process could be [sidekiq](http://sidekiq.org/). Environment variables are sometimes different also. The important key is that the same docker image is used for all 3 services but the task definition for each service is different.
|
@@ -186,7 +183,7 @@ This is easily accomplished with the `bin/deploy` wrapper script that the `ufo i
|
|
186
183
|
|
187
184
|
ufo ship hi-worker --cluster stag --no-wait
|
188
185
|
ufo ship hi-clock --cluster stag --no-wait --no-docker
|
189
|
-
ufo ship hi-web
|
186
|
+
ufo ship hi-web-prod--cluster stag --no-docker
|
190
187
|
```
|
191
188
|
|
192
189
|
The first `ufo ship hi-worker` command build and ships docker image to ECS, but the following two `ufo ship` commands use the `--no-docker` flag to skip the `docker build` step. `ufo ship` will use the last built docker image as the image to be shipped. For those curious, this is stored in `ufo/docker_image_name_ufo.txt`.
|
@@ -196,7 +193,7 @@ The first `ufo ship hi-worker` command build and ships docker image to ECS, but
|
|
196
193
|
Ufo assumes a convention that service\_name and the task\_name are the same. If you would like to override this convention then you can specify the task name.
|
197
194
|
|
198
195
|
```
|
199
|
-
ufo ship hi-web
|
196
|
+
ufo ship hi-web-prod--task my-task
|
200
197
|
```
|
201
198
|
|
202
199
|
This means that in the task_definition.rb you will also defined it with `my-task`. For example:
|
@@ -233,21 +230,40 @@ ufo tasks register # will register all genreated task definitinos in the ufo/out
|
|
233
230
|
Skips all the build docker phases of a deploy sequence and only update the service with the task definitions.
|
234
231
|
|
235
232
|
```bash
|
236
|
-
ufo ship hi-web
|
233
|
+
ufo ship hi-web-prod--no-docker
|
237
234
|
```
|
238
235
|
Note if you use the `--no-docker` option you should ensure that you have already push a docker image to your docker register. Or else the task will not be able to spin up because the docker image does not exist. I recommend that you normally use `ufo ship` most of the time.
|
239
236
|
|
240
237
|
|
241
|
-
|
238
|
+
### Automated Docker Images Clean Up
|
242
239
|
|
243
240
|
Ufo can be configured to automatically clean old images from the ECR registry after the deploy completes. I normally set `~/.ufo/settings.yml` like so:
|
244
241
|
|
245
242
|
```yaml
|
246
|
-
ecr_keep:
|
243
|
+
ecr_keep: 30
|
247
244
|
```
|
248
245
|
|
249
246
|
Automated Docker images clean up only works if you are using ECR registry.
|
250
247
|
|
248
|
+
## Running a single task
|
249
|
+
|
250
|
+
Sometimes you do not want to run a long running `service` but a one time task. Running Rails migrations are a good example of a one off task. Here is an example of how you would run a one time task.
|
251
|
+
|
252
|
+
```
|
253
|
+
ufo task hi-migrate-prod
|
254
|
+
```
|
255
|
+
|
256
|
+
You will need to define a task definition for the migrate command also in `ufo/task_definitions.rb`. If you only need to override the Docker command and can re-use an existing task definition like `hi-web-prod`. You can use the `--override-command` option:
|
257
|
+
|
258
|
+
```
|
259
|
+
ufo task hi-web-prod --override-command '["bin/migrate"]'
|
260
|
+
ufo task hi-web-prod --override-command bin/migrate
|
261
|
+
ufo task hi-web-prod --override-command "bin/with_env bundle exec rake db:migrate:redo VERSION=xxx"
|
262
|
+
ufo task hi-web-prod --override-command '["bin/with_env", "bundle", "exec", "rake", "db:migrate:redo", "VERSION=xxx"]''
|
263
|
+
```
|
264
|
+
|
265
|
+
The `--override-command` option takes a string. If the string has brackets in it then it will be evaluated as an Array but the option must be a string.
|
266
|
+
|
251
267
|
## Scale
|
252
268
|
|
253
269
|
There is a convenience wrapper that simple executes `aws ecs update-service --service [SERVICE] --desired-count [COUNT]`
|
@@ -13,6 +13,11 @@ common = {
|
|
13
13
|
cpu: 128,
|
14
14
|
memory_reservation: 256,
|
15
15
|
environment: helper.env_file(".env")
|
16
|
+
# another example
|
17
|
+
# environment: helper.env_vars(%Q{
|
18
|
+
# RAILS_ENV=production
|
19
|
+
# SECRET_KEY_BASE=secret
|
20
|
+
# })
|
16
21
|
}
|
17
22
|
|
18
23
|
task_definition "<%= @app %>-web" do
|
data/lib/ufo.rb
CHANGED
@@ -17,6 +17,7 @@ module Ufo
|
|
17
17
|
autoload :TasksBuilder, 'ufo/tasks_builder'
|
18
18
|
autoload :TasksRegister, 'ufo/tasks_register'
|
19
19
|
autoload :Ship, 'ufo/ship'
|
20
|
+
autoload :Task, 'ufo/task'
|
20
21
|
autoload :EcrCleaner, 'ufo/ecr_cleaner'
|
21
22
|
autoload :Destroy, 'ufo/destroy'
|
22
23
|
autoload :Scale, 'ufo/scale'
|
data/lib/ufo/cli.rb
CHANGED
@@ -95,23 +95,12 @@ module Ufo
|
|
95
95
|
option :ecr_keep, type: :numeric, desc: "ECR specific cleanup of old images. Specifies how many images to keep. Only runs if the images are ECR images. Defaults to keeping all the images."
|
96
96
|
long_desc Help.ship
|
97
97
|
def ship(service)
|
98
|
-
builder =
|
99
|
-
if options[:docker]
|
100
|
-
builder.build
|
101
|
-
builder.push
|
102
|
-
end
|
103
|
-
|
104
|
-
# task definition and deploy logic are coupled in the Ship class.
|
105
|
-
# Example: We need to know if the task defintion is a web service to see if we need to
|
106
|
-
# add the elb target group. The web service information is in the TasksBuilder
|
107
|
-
# and the elb target group gets set in the Ship class.
|
108
|
-
# So we always call these together.
|
109
|
-
TasksBuilder.new(options).build
|
98
|
+
builder = build_docker(options)
|
110
99
|
task_definition = options[:task] || service # convention
|
111
|
-
|
112
|
-
ship
|
100
|
+
register_task(task_definition, options)
|
101
|
+
return if ENV['TEST'] # allows quick testing of the ship CLI portion only
|
113
102
|
|
114
|
-
|
103
|
+
ship = Ship.new(service, task_definition, options)
|
115
104
|
ship.deploy
|
116
105
|
if options[:docker]
|
117
106
|
DockerCleaner.new(builder.image_name, options).cleanup
|
@@ -120,6 +109,16 @@ module Ufo
|
|
120
109
|
puts "Docker image shipped: #{builder.full_image_name.green}"
|
121
110
|
end
|
122
111
|
|
112
|
+
desc "task [TASK_DEFINITION]", "runs a one time task"
|
113
|
+
long_desc Help.task
|
114
|
+
option :docker, type: :boolean, desc: "Enable docker build and push", default: true
|
115
|
+
option :command, desc: "Override the command used for the container"
|
116
|
+
def task(task_definition)
|
117
|
+
build_docker(options)
|
118
|
+
register_task(task_definition, options)
|
119
|
+
Task.new(task_definition, options).run
|
120
|
+
end
|
121
|
+
|
123
122
|
desc "destroy [SERVICE]", "destroys the ECS service"
|
124
123
|
long_desc Help.destroy
|
125
124
|
option :force, type: :boolean, desc: "By pass are you sure prompt."
|
@@ -133,5 +132,31 @@ module Ufo
|
|
133
132
|
def scale(service, count)
|
134
133
|
Scale.new(service, count, options).update
|
135
134
|
end
|
135
|
+
|
136
|
+
desc "version", "Prints version number of installed ufo"
|
137
|
+
def version
|
138
|
+
puts Ufo::VERSION
|
139
|
+
end
|
140
|
+
|
141
|
+
no_tasks do
|
142
|
+
def build_docker(options)
|
143
|
+
builder = DockerBuilder.new(options) # outside if because it need docker.full_image_name
|
144
|
+
if options[:docker]
|
145
|
+
builder.build
|
146
|
+
builder.push
|
147
|
+
end
|
148
|
+
builder
|
149
|
+
end
|
150
|
+
|
151
|
+
def register_task(task_definition, options)
|
152
|
+
# task definition and deploy logic are coupled in the Ship class.
|
153
|
+
# Example: We need to know if the task defintion is a web service to see if we need to
|
154
|
+
# add the elb target group. The web service information is in the TasksBuilder
|
155
|
+
# and the elb target group gets set in the Ship class.
|
156
|
+
# So we always call these together.
|
157
|
+
TasksBuilder.new(options).build
|
158
|
+
TasksRegister.register(task_definition, options)
|
159
|
+
end
|
160
|
+
end
|
136
161
|
end
|
137
162
|
end
|
data/lib/ufo/cli/help.rb
CHANGED
@@ -137,6 +137,28 @@ $ ufo ship hi-web-prod --no-elb-prompt
|
|
137
137
|
EOL
|
138
138
|
end
|
139
139
|
|
140
|
+
def task
|
141
|
+
<<-EOL
|
142
|
+
Examples:
|
143
|
+
|
144
|
+
To run a one time task with ECS:
|
145
|
+
|
146
|
+
$ ufo task hi-migrate-prod
|
147
|
+
|
148
|
+
You can also override the command used by the Docker container in the task definitions via override-command.
|
149
|
+
|
150
|
+
ufo task hi-web-prod --override-command bin/migrate
|
151
|
+
|
152
|
+
ufo task hi-web-prod --override-command '["bin/migrate"]'
|
153
|
+
|
154
|
+
ufo task hi-web-prod --override-command "bin/with_env bundle exec rake db:migrate:redo VERSION=xxx"
|
155
|
+
|
156
|
+
ufo task hi-web-prod --override-command '["bin/with_env","bundle","exec","rake","db:migrate:redo","VERSION=xxx"]''
|
157
|
+
|
158
|
+
The `--override-command` option takes a string. If the string has brackets in it then it will be evaluated as an Array but the optoin must be a string. Also there can be no spaces in the string.
|
159
|
+
EOL
|
160
|
+
end
|
161
|
+
|
140
162
|
def destroy
|
141
163
|
<<-EOL
|
142
164
|
Examples:
|
data/lib/ufo/settings.rb
CHANGED
@@ -15,7 +15,7 @@ module Ufo
|
|
15
15
|
@data = YAML.load_file(settings_path)
|
16
16
|
@data = user_settings.merge(@data)
|
17
17
|
else
|
18
|
-
puts "ERROR: No settings file
|
18
|
+
puts "ERROR: No settings file at #{settings_path}"
|
19
19
|
puts "Please create a settings file via: ufo init"
|
20
20
|
exit 1
|
21
21
|
end
|
data/lib/ufo/ship.rb
CHANGED
@@ -75,23 +75,6 @@ module Ufo
|
|
75
75
|
stop_old_task(deployed_service) if @stop_old_tasks
|
76
76
|
end
|
77
77
|
|
78
|
-
def process_multiple_services
|
79
|
-
puts "Multi services mode" unless @options[:mute]
|
80
|
-
services_to_deploy = []
|
81
|
-
find_all_ecs_services do |ecs_service|
|
82
|
-
if service_pattern_match?(ecs_service.service_name)
|
83
|
-
services_to_deploy << ecs_service
|
84
|
-
end
|
85
|
-
end
|
86
|
-
|
87
|
-
deployed_services = services_to_deploy.map do |ecs_service|
|
88
|
-
update_service(ecs_service)
|
89
|
-
end
|
90
|
-
|
91
|
-
wait_for_all_deployments(deployed_services) if @wait_for_deployment && !@options[:noop]
|
92
|
-
stop_old_tasks(deployed_services) if @stop_old_tasks
|
93
|
-
end
|
94
|
-
|
95
78
|
def service_tasks(cluster, service)
|
96
79
|
all_task_arns = ecs.list_tasks(cluster: cluster, service_name: service).task_arns
|
97
80
|
return [] if all_task_arns.empty?
|
@@ -367,19 +350,6 @@ module Ufo
|
|
367
350
|
}
|
368
351
|
end
|
369
352
|
|
370
|
-
def service_exact_match?(service_name)
|
371
|
-
service_name == @service
|
372
|
-
end
|
373
|
-
|
374
|
-
# @service is changed to a pattern to be search with.
|
375
|
-
#
|
376
|
-
# Examples:
|
377
|
-
# @service == "hi-.*-prod"
|
378
|
-
def service_pattern_match?(service_name)
|
379
|
-
service_patttern = Regexp.new(@service)
|
380
|
-
service_name =~ service_patttern
|
381
|
-
end
|
382
|
-
|
383
353
|
def find_ecs_service
|
384
354
|
find_all_ecs_services.find { |ecs_service| ecs_service.service_name == @service }
|
385
355
|
end
|
data/lib/ufo/task.rb
ADDED
@@ -0,0 +1,53 @@
|
|
1
|
+
module Ufo
|
2
|
+
class Task
|
3
|
+
include Defaults
|
4
|
+
include AwsServices
|
5
|
+
|
6
|
+
def initialize(task_definition, options)
|
7
|
+
@task_definition = task_definition
|
8
|
+
@options = options
|
9
|
+
@cluster = @options[:cluster] || default_cluster
|
10
|
+
end
|
11
|
+
|
12
|
+
def run
|
13
|
+
puts "Running task_definition: #{@task_definition}".colorize(:green) unless @options[:mute]
|
14
|
+
return if @options[:noop]
|
15
|
+
task_options = {
|
16
|
+
cluster: @cluster,
|
17
|
+
task_definition: @task_definition
|
18
|
+
}
|
19
|
+
task_options.merge!(overrides: overrides) if @options[:command]
|
20
|
+
resp = ecs.run_task(task_options)
|
21
|
+
puts "Task ARN: #{resp.tasks[0].task_arn}" unless @options[:mute]
|
22
|
+
end
|
23
|
+
|
24
|
+
private
|
25
|
+
# only using the overrides to override the container command
|
26
|
+
def overrides
|
27
|
+
command = @options[:command]
|
28
|
+
command = eval(command) if command.include?('[') # command is in Array form
|
29
|
+
container_definition = get_original_container_definition
|
30
|
+
{
|
31
|
+
container_overrides: [
|
32
|
+
{
|
33
|
+
name: container_definition[:name],
|
34
|
+
command: command,
|
35
|
+
environment: container_definition[:environment],
|
36
|
+
},
|
37
|
+
]
|
38
|
+
}
|
39
|
+
end
|
40
|
+
|
41
|
+
def get_original_container_definition
|
42
|
+
resp = ecs.list_task_definitions(
|
43
|
+
family_prefix: @task_definition,
|
44
|
+
sort: "DESC"
|
45
|
+
)
|
46
|
+
# "arn:aws:ecs:us-east-1:<aws_account_id>:task-definition/wordpress:6",
|
47
|
+
last_definition_arn = resp.task_definition_arns.first
|
48
|
+
task_name = last_definition_arn.split("/").last
|
49
|
+
resp = ecs.describe_task_definition(task_definition: task_name)
|
50
|
+
container_definition = resp.task_definition.container_definitions[0].to_h
|
51
|
+
end
|
52
|
+
end
|
53
|
+
end
|
data/lib/ufo/version.rb
CHANGED
data/spec/lib/cli_spec.rb
CHANGED
@@ -46,5 +46,12 @@ describe Ufo::CLI do
|
|
46
46
|
expect(out).to include("Task Definitions built")
|
47
47
|
end
|
48
48
|
end
|
49
|
+
|
50
|
+
context "task" do
|
51
|
+
it "runs one time task" do
|
52
|
+
out = execute("bin/ufo task hi-migrate-prod #{@args}")
|
53
|
+
expect(out).to include("Running task_definition")
|
54
|
+
end
|
55
|
+
end
|
49
56
|
end
|
50
57
|
end
|
@@ -3,7 +3,13 @@ require 'spec_helper'
|
|
3
3
|
describe Ufo::EcrCleaner do
|
4
4
|
let(:docker_image_name) { "123456789.dkr.ecr.us-east-1.amazonaws.com/my-name" }
|
5
5
|
let(:repo_domain) { "https://123456789.dkr.ecr.us-east-1.amazonaws.com" }
|
6
|
-
let(:cleaner)
|
6
|
+
let(:cleaner) do
|
7
|
+
Ufo::EcrCleaner.new(docker_image_name,
|
8
|
+
project_root: "spec/fixtures/hi",
|
9
|
+
ecr_keep: 3, # using 3 to test, default is 30
|
10
|
+
mute: true
|
11
|
+
)
|
12
|
+
end
|
7
13
|
before(:each) do
|
8
14
|
allow(cleaner).to receive(:update_auth_token).and_return(:whatever)
|
9
15
|
end
|
data/spec/lib/ship_spec.rb
CHANGED
@@ -11,14 +11,20 @@ describe Ufo::Ship do
|
|
11
11
|
stop_old_tasks: false,
|
12
12
|
}
|
13
13
|
end
|
14
|
+
let(:service) { "hi-web-prod" }
|
14
15
|
let(:task_definition) { service }
|
15
|
-
let(:ship)
|
16
|
+
let(:ship) do
|
17
|
+
ship = Ufo::Ship.new(service, task_definition, options)
|
18
|
+
allow(ship).to receive(:ecs).and_return(ecs_client)
|
19
|
+
ship
|
20
|
+
end
|
16
21
|
|
17
|
-
context "
|
18
|
-
|
19
|
-
it "should create or update single service" do
|
22
|
+
context "hi-web-prod service" do
|
23
|
+
it "should create or update service" do
|
20
24
|
allow(ship).to receive(:process_single_service)
|
25
|
+
|
21
26
|
ship.deploy
|
27
|
+
|
22
28
|
expect(ship).to have_received(:process_single_service)
|
23
29
|
end
|
24
30
|
|
@@ -26,49 +32,37 @@ describe Ufo::Ship do
|
|
26
32
|
it "should create service on first cluster" do
|
27
33
|
allow(ship).to receive(:find_ecs_service)
|
28
34
|
allow(ship).to receive(:create_service)
|
35
|
+
|
29
36
|
ship.deploy
|
37
|
+
|
30
38
|
expect(ship).to have_received(:create_service)
|
31
39
|
end
|
32
40
|
end
|
33
41
|
|
34
42
|
context "1 services found" do
|
35
|
-
it "should call update service
|
43
|
+
it "should call update service" do
|
36
44
|
allow(ship).to receive(:find_ecs_service).and_return(ecs_service("hi-web-prod"))
|
37
45
|
allow(ship).to receive(:update_service)
|
46
|
+
|
38
47
|
ship.deploy
|
48
|
+
|
39
49
|
expect(ship).to have_received(:update_service).exactly(1).times
|
40
50
|
end
|
41
51
|
end
|
42
52
|
end
|
43
53
|
|
44
|
-
|
45
|
-
|
46
|
-
|
47
|
-
|
48
|
-
|
49
|
-
|
50
|
-
end
|
51
|
-
|
52
|
-
context "0 services found" do
|
53
|
-
it "should not call update_service" do
|
54
|
-
allow(ship).to receive(:find_all_ecs_services)
|
55
|
-
allow(ship).to receive(:update_service)
|
56
|
-
ship.deploy
|
57
|
-
expect(ship).to_not have_received(:update_service)
|
58
|
-
end
|
59
|
-
end
|
54
|
+
# mocks
|
55
|
+
def ecs_client
|
56
|
+
ecs = double("ecs")
|
57
|
+
allow(ecs).to receive(:describe_clusters).and_return(ecs_describe_clusters)
|
58
|
+
ecs
|
59
|
+
end
|
60
60
|
|
61
|
-
|
62
|
-
|
63
|
-
|
64
|
-
|
65
|
-
|
66
|
-
and_yield(ecs_service("hi-clock-prod"))
|
67
|
-
allow(ship).to receive(:update_service)
|
68
|
-
ship.deploy
|
69
|
-
expect(ship).to have_received(:update_service).exactly(3).times
|
70
|
-
end
|
71
|
-
end
|
61
|
+
# ensure_cluster_exist calls this and this makes sure that the cluster 'exists'
|
62
|
+
def ecs_describe_clusters
|
63
|
+
describe_clusters = double("ecs-describe-clusters")
|
64
|
+
allow(describe_clusters).to receive(:clusters).and_return(["cluster1"])
|
65
|
+
describe_clusters
|
72
66
|
end
|
73
67
|
|
74
68
|
def ecs_service(name)
|
@@ -0,0 +1,33 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
|
3
|
+
describe Ufo::Ship do
|
4
|
+
let(:project_root) { File.expand_path("../../fixtures/hi", __FILE__) }
|
5
|
+
let(:options) do
|
6
|
+
{
|
7
|
+
project_root: project_root,
|
8
|
+
mute: true
|
9
|
+
}
|
10
|
+
end
|
11
|
+
let(:task_definition) { "hi-migrate-prod" }
|
12
|
+
let(:task) do
|
13
|
+
task = Ufo::Task.new(task_definition, options)
|
14
|
+
allow(task).to receive(:ecs).and_return(ecs_client)
|
15
|
+
task
|
16
|
+
end
|
17
|
+
|
18
|
+
context "hi-migrate-prod" do
|
19
|
+
it "should migrate the database" do
|
20
|
+
task.run
|
21
|
+
|
22
|
+
expect(task.ecs).to have_received(:run_task)
|
23
|
+
end
|
24
|
+
end
|
25
|
+
|
26
|
+
# mocks
|
27
|
+
def ecs_client
|
28
|
+
ecs = double("ecs")
|
29
|
+
fake_response = double('fake-response').as_null_object
|
30
|
+
allow(ecs).to receive(:run_task).and_return(fake_response)
|
31
|
+
ecs
|
32
|
+
end
|
33
|
+
end
|
data/spec/spec_helper.rb
CHANGED
@@ -11,7 +11,7 @@ require "#{root}/lib/ufo"
|
|
11
11
|
|
12
12
|
module Helpers
|
13
13
|
def execute(cmd)
|
14
|
-
puts "Running: #{cmd}" if ENV['DEBUG']
|
14
|
+
puts "Running: #{cmd.colorize(:magenta)}" if ENV['DEBUG']
|
15
15
|
out = `#{cmd}`
|
16
16
|
puts out if ENV['DEBUG']
|
17
17
|
out
|
@@ -25,4 +25,9 @@ end
|
|
25
25
|
|
26
26
|
RSpec.configure do |c|
|
27
27
|
c.include Helpers
|
28
|
+
|
29
|
+
c.before(:each) do
|
30
|
+
# ensures we never called real aws api since the fixture home folder does not have ~/.aws/credentails setup
|
31
|
+
ENV['HOME'] = "spec/fixtures/home"
|
32
|
+
end
|
28
33
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: ufo
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 1.0.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Tung Nguyen
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2017-
|
11
|
+
date: 2017-05-08 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: thor
|
@@ -223,6 +223,7 @@ files:
|
|
223
223
|
- lib/ufo/scale.rb
|
224
224
|
- lib/ufo/settings.rb
|
225
225
|
- lib/ufo/ship.rb
|
226
|
+
- lib/ufo/task.rb
|
226
227
|
- lib/ufo/tasks_builder.rb
|
227
228
|
- lib/ufo/tasks_register.rb
|
228
229
|
- lib/ufo/templates/default.json.erb
|
@@ -232,6 +233,7 @@ files:
|
|
232
233
|
- spec/lib/ecr_auth_spec.rb
|
233
234
|
- spec/lib/ecr_cleaner_spec.rb
|
234
235
|
- spec/lib/ship_spec.rb
|
236
|
+
- spec/lib/task_spec.rb
|
235
237
|
- spec/spec_helper.rb
|
236
238
|
- ufo.gemspec
|
237
239
|
homepage: https://github.com/tongueroo/ufo
|
@@ -254,7 +256,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
254
256
|
version: '0'
|
255
257
|
requirements: []
|
256
258
|
rubyforge_project:
|
257
|
-
rubygems_version: 2.
|
259
|
+
rubygems_version: 2.5.2
|
258
260
|
signing_key:
|
259
261
|
specification_version: 4
|
260
262
|
summary: Build Docker Containers and Ship Them to AWS ECS
|
@@ -264,4 +266,5 @@ test_files:
|
|
264
266
|
- spec/lib/ecr_auth_spec.rb
|
265
267
|
- spec/lib/ecr_cleaner_spec.rb
|
266
268
|
- spec/lib/ship_spec.rb
|
269
|
+
- spec/lib/task_spec.rb
|
267
270
|
- spec/spec_helper.rb
|