ufo 0.1.6 → 1.0.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 47240674a8c9844d9abf4f3f7541f63afbfde413
4
- data.tar.gz: 6bb1a923819cd3381e11843c9916410efb50ea7c
3
+ metadata.gz: dfca9e6d1cee650f151b4ad478f5fd3fb48ab840
4
+ data.tar.gz: 76d021b7c76d3215fe4fbddd07785c26d889c31c
5
5
  SHA512:
6
- metadata.gz: 0efa6e8127ab99e3ba621f7edd3bf75bede2c4c64d4d2f80efa8bc82daec6ad45f69e9c9695a3014dc0e97408e5753911492a279226c74547fda74629713ef3f
7
- data.tar.gz: a43d4e17f621b7ff18fe5c18c015a417dec30dd51dac9ed1ae07abbdffdb987b63a05c177d4c4271f528e377db3a45fd35f7164a11094725b38b61c36c12ac80
6
+ metadata.gz: 23b34f792eeff2b250a5215a2b44c99405aa04cc33bc985c1f5ccbc6ee66587b25bc98f869e4cc3f8c03a4710a38db5147457c606cdcafedc9ca700ea420cdee
7
+ data.tar.gz: 5eb6c95ab9839e658f9cd4dae7275d4d1bf0c6c784f92d169e4c036e85fd341b0a36d6412707dd0e681a7978c2f2553c12d6cd9b1644650896423326b8ee310a
@@ -3,6 +3,11 @@
3
3
  All notable changes to this project will be documented in this file.
4
4
  This project *tries* to adhere to [Semantic Versioning](http://semver.org/), even before v1.0.
5
5
 
6
+ ## [1.0.0]
7
+
8
+ * add `ufo task` command
9
+ * been using in production for a while, ready for 1.0.0 release
10
+
6
11
  ## [0.1.6]
7
12
 
8
13
  * default wait for deployment to false
data/Gemfile CHANGED
@@ -4,4 +4,3 @@ source 'https://rubygems.org'
4
4
  gemspec
5
5
 
6
6
  gem "codeclimate-test-reporter", group: :test, require: nil
7
- gem "byebug"
data/README.md CHANGED
@@ -1,4 +1,6 @@
1
- # Ufo - Build Docker Containers and Ship Them to AWS ECS
1
+ # Ufo - A Easy Way to Build and Ship Docker Images AWS ECS
2
+
3
+ [![CircleCI](https://circleci.com/gh/tongueroo/ufo.svg?style=svg)](https://circleci.com/gh/tongueroo/ufo)
2
4
 
3
5
  ## Quick Introduction
4
6
 
@@ -7,11 +9,11 @@ Ufo is a simple tool that makes building and shipping Docker containers to [AWS
7
9
  * This blog post provides an introduction to the tool: [Ufo - Build Docker Containers and Ship Them to AWS ECS](https://medium.com/@tongueroo/ufo-easily-build-docker-containers-and-ship-them-to-aws-ecs-15556a2b39f#.qqu8o4wal).
8
10
  * This presentation covers ufo also: [Ufo Ship on AWS ECS](http://www.slideshare.net/tongueroo/ufo-ship-for-aws-ecs-70885296)
9
11
 
10
- A summary of steps `ufo ship` takes:
12
+ A summary of what `ufo ship` does:
11
13
 
12
- 1. builds a docker image. 
13
- 2. generates and registers the ECS template definition. 
14
- 3. deploys the ECS template definition to the specified service.
14
+ 1. Builds a docker image. 
15
+ 2. Generates and registers the ECS template definition. 
16
+ 3. Deploys the ECS template definition to the specified service.
15
17
 
16
18
  Ufo deploys a task definition that is created via a template generator which is fully controllable.
17
19
 
@@ -20,13 +22,46 @@ Ufo deploys a task definition that is created via a template generator which is
20
22
 
21
23
  $ gem install ufo
22
24
 
23
- You will need a working version of docker installed as ufo calls the docker command.
25
+ Dependencies: You will need a working version of docker installed as ufo shells out and calls the `docker` command.
26
+
27
+ ## Quick Usage
28
+
29
+ ### ufo ship
24
30
 
25
- ## Usage
31
+ To execute the ship process run:
26
32
 
27
- When using ufo if the ECS service does not yet exist, it will automatically be created for you. If you are relying on this tool to create the cluster, you still need to associate ECS Container Instances to the cluster yourself.
33
+ ```bash
34
+ ufo ship hi-web-prod--cluster mycluster
35
+ ```
28
36
 
29
- First initialize ufo files within your project. Let's say you have an `hi` app.
37
+ Note, if you have configured `ufo/settings.yml` to map hi-web-prod to the `mycluster` cluster using the service_cluster option the command becomes simply:
38
+
39
+ ```bash
40
+ ufo ship hi-web-prod
41
+ ```
42
+
43
+ When you run `ufo ship hi-web-prod`:
44
+
45
+ 1. It builds the docker image.
46
+ 2. Generates a task definition and registers it.
47
+ 3. Updates the ECS service to use it.
48
+
49
+ If the ECS service hi-web-prod does not yet exist, ufo will create the service for you.
50
+
51
+ By convention, if the service has a container name web, you'll get prompted to create an ELB and specify a target group arn. The ELB and target group must already exist.
52
+
53
+ You can bypass the prompt and specify the target group arn as part of the command. The elb target group can only be associated when the service gets created for the first time. If the service already exists then the `--target-group` parameter just gets ignored and the ECS task simply gets updated. Example:
54
+
55
+
56
+ ```bash
57
+ ufo ship hi-web-prod --target-group=arn:aws:elasticloadbalancing:us-east-1:12345689:targetgroup/hi-web-prod/12345
58
+ ```
59
+
60
+ When using ufo if the ECS service does not yet exist, it will automatically be created for you. Ufo will also automatically create the ECS cluster. If you are relying on this tool to create the cluster, you still need to associate ECS Container Instances to the cluster yourself.
61
+
62
+ ## Detailed Usage
63
+
64
+ First initialize ufo files within your project. Let's say you have an example `hi` app.
30
65
 
31
66
  ```
32
67
  $ git clone https://github.com/tongueroo/hi
@@ -43,30 +78,30 @@ Starter ufo files created.
43
78
  $
44
79
  ```
45
80
 
46
- Take a look at the `ufo/settings.yml` file to see that it holds some default configuration settings so you don't have to type out these options every single time.
81
+ Take a look at the `ufo/settings.yml` file and notice that it contains some default configuration settings so you do not have to type out these options repeatedly for some of the ufo commands.
47
82
 
48
83
  ```yaml
49
84
  image: tongueroo/hi
50
85
  service_cluster:
51
- default: stag # default cluster
52
- hi-web: stag
53
- hi-clock: stag
54
- hi-worker: stag
86
+ default: prod # default cluster
87
+ hi-web-prod: blue
88
+ hi-clock-prod: blue
89
+ hi-worker-prod: blue
55
90
  ```
56
91
 
57
92
  The `image` value is the name that ufo will use for the Docker image name.
58
93
 
59
- The `service_cluster` mapping provides a way to set default service to cluster mappings so that you do not have to specify the `--cluster` repeatedly. Example:
94
+ The `service_cluster` mapping provides a way to set default service-to-cluster mappings so that you do not have to specify the `--cluster` repeatedly. This is very helpful. For example:
60
95
 
61
96
  ```
62
- ufo ship hi-web --cluster hi-cluster
63
- ufo ship hi-web # same as above because it is configured in ufo/settings.yml
64
- ufo ship hi-web --cluster special-cluster # overrides any setting default fallback.
97
+ ufo ship hi-web-prod --cluster hi-cluster
98
+ ufo ship hi-web-prod # same as above because it is configured in ufo/settings.yml
99
+ ufo ship hi-web-prod --cluster special-cluster # overrides the default setting in `ufo/settings.yml`.
65
100
  ```
66
101
 
67
- ## Task Definition ERB Template and DSL Generator
102
+ ### Task Definition ERB Template and DSL Generator
68
103
 
69
- Ufo task definitions are is written in a template generator DSL to provide full control of the task definition that gets uploaded for each service. We'll go over a simple example. Here is the ERB template for `ufo/templates/main.json.erb`:
104
+ Ufo task definitions are written as an ERB template that makes it every easily accessible and configurable to your requirements. Here is is an example of an ERB template `ufo/templates/main.json.erb` that shows how easy it is to modfied the task definition you want to be uploaded by ufo:
70
105
 
71
106
  ```json
72
107
  {
@@ -110,16 +145,16 @@ Ufo task definitions are is written in a template generator DSL to provide full
110
145
  }
111
146
  ```
112
147
 
113
- The instance variable values are specified in `ufo/task_definitions.rb`. Here's the
148
+ The instance variable values are specified in `ufo/task_definitions.rb` via a DSL. Here's the
114
149
 
115
150
 
116
151
  ```ruby
117
- task_definition "hi-web" do
152
+ task_definition "hi-web-prod" do
118
153
  source "main" # will use ufo/templates/main.json.erb
119
154
  variables(
120
155
  family: task_definition_name,
121
156
  # image: tongueroo/hi:ufo-[timestamp]=[sha]
122
- image: helper.full_image_name,
157
+ image: helper.full_image_name,
123
158
  environment: env_file('.env.prod')
124
159
  name: "web",
125
160
  container_port: helper.dockerfile_port,
@@ -128,53 +163,15 @@ task_definition "hi-web" do
128
163
  end
129
164
  ```
130
165
 
131
- As you can see above, the task\_definitions.rb file has some special variables and helper methods available. These helper methods provide useful contextual information about the project. For example, one of the variable provides the exposed port in the Dockerfile of the project. Here is a list of the important ones:
166
+ The task\_definitions.rb file has some special variables and helper methods available. These helper methods provide useful contextual information about the project. For example, one of the variable provides the exposed port in the Dockerfile of the project. This why if someone changes the exported port in the Dockerfile, he will not "forgot" to also update the ufo variable as it is automatically referenced. Here is a list of the important helpers:
132
167
 
133
- * **helper.full\_image\_name** — The full docker image name that ufo builds. The “base” portion of the docker image name is defined in ufo/settings.yml. For example, the base portion is "tongueroo/hi" and the full image name is tongueroo/hi:ufo-[timestamp]-[sha]. So the base does not include the Docker tag and the full image name does include the tag.
168
+ * **helper.full\_image\_name** — The full docker image name that ufo builds. The “base” portion of the docker image name is defined in ufo/settings.yml. For example, the base portion is `tongueroo/hi` and the full image name is `tongueroo/hi:ufo-[timestamp]-[sha]`. The base name does not include the generated Docker tag, which contains a timestamp and git sha of the Dockerfile that is used.
134
169
  * **helper.dockerfile\_port** — Exposed port extracted from the Dockerfile of the project. 
135
- * **env_file** — This method takes an .env file which contains a simple key value list of environment variables and converts the list to the proper task definition json format.
170
+ * **env_vars(text)** — This method takes a block of text that contains the env values in key=value format and converts that block of text to the proper task definition json format.
171
+ * **env_file(path)** — This method takes an `.env` file which contains a simple key value list of environment variables and converts the list to the proper task definition json format.
136
172
 
137
173
  The 2 classes which provide these special helper methods are in [ufo/dsl.rb](https://github.com/tongueroo/ufo/blob/master/lib/ufo/dsl.rb) and [ufo/dsl/helper.rb](https://github.com/tongueroo/ufo/blob/master/lib/ufo/dsl/helper.rb). Refer to these classes for the full list of the special variables and methods.
138
174
 
139
- ### Customizing Templates
140
-
141
- If you want to change the template then you can follow the example in the generated ufo files. For example, if you want to create a template for the worker service.
142
-
143
- 1. Create a new template under ufo/templates/worker.json.erb.
144
- 2. Change the source in the `task_definition` using "worker" as the source.
145
- 3. Add variables.
146
-
147
- ### ufo ship
148
-
149
- Ufo uses the aforementioned files to build task definitions and then ship to them to AWS ECS. To execute the ship process run:
150
-
151
- ```bash
152
- ufo ship hi-web --cluster stag
153
- ```
154
-
155
- Note, if you have configured `ufo/settings.yml` to map hi-web to the stag cluster using the service_cluster option the command becomes simply:
156
-
157
- ```bash
158
- ufo ship hi-web
159
- ```
160
-
161
- When you run `ufo ship hi-web`:
162
-
163
- 1. It builds the docker image.
164
- 2. Generates a task definition and registers it.
165
- 3. Updates the ECS service to use it.
166
-
167
- If the ECS service hi-web does not yet exist, ufo will create the service for you.
168
-
169
- If the service has a container name web, you'll get prompted to create an ELB and specify a target group arn. The ELB and target group must already exist.
170
-
171
- You can bypass the prompt and specify the target group arn as part of the command. The elb target group can only be associated when the service gets created for the first time. If the service already exists then the `--target-group` parameter just gets ignored and the ECS task simply gets updated. Example:
172
-
173
-
174
- ```bash
175
- ufo ship hi-web --target-group=arn:aws:elasticloadbalancing:us-east-1:12345689:targetgroup/hi-web/jdisljflsdkjl
176
- ```
177
-
178
175
  ### Shipping Multiple Services with bin/deploy
179
176
 
180
177
  A common pattern is to have 3 processes: web, worker, and clock. This is very common in rails applcations. The web process handles web traffic, the worker process handles background job processing that would be too slow and potentially block web requests, and a clock process is typically used to schedule recurring jobs. These processes use the same codebase, or same docker image, but have slightly different run time settings. For example, the docker run command for a web process could be [puma](http://puma.io/) and the command for a worker process could be [sidekiq](http://sidekiq.org/). Environment variables are sometimes different also. The important key is that the same docker image is used for all 3 services but the task definition for each service is different.
@@ -186,7 +183,7 @@ This is easily accomplished with the `bin/deploy` wrapper script that the `ufo i
186
183
 
187
184
  ufo ship hi-worker --cluster stag --no-wait
188
185
  ufo ship hi-clock --cluster stag --no-wait --no-docker
189
- ufo ship hi-web --cluster stag --no-docker
186
+ ufo ship hi-web-prod--cluster stag --no-docker
190
187
  ```
191
188
 
192
189
  The first `ufo ship hi-worker` command build and ships docker image to ECS, but the following two `ufo ship` commands use the `--no-docker` flag to skip the `docker build` step. `ufo ship` will use the last built docker image as the image to be shipped. For those curious, this is stored in `ufo/docker_image_name_ufo.txt`.
@@ -196,7 +193,7 @@ The first `ufo ship hi-worker` command build and ships docker image to ECS, but
196
193
  Ufo assumes a convention that service\_name and the task\_name are the same. If you would like to override this convention then you can specify the task name.
197
194
 
198
195
  ```
199
- ufo ship hi-web --task my-task
196
+ ufo ship hi-web-prod--task my-task
200
197
  ```
201
198
 
202
199
  This means that in the task_definition.rb you will also defined it with `my-task`. For example:
@@ -233,21 +230,40 @@ ufo tasks register # will register all genreated task definitinos in the ufo/out
233
230
  Skips all the build docker phases of a deploy sequence and only update the service with the task definitions.
234
231
 
235
232
  ```bash
236
- ufo ship hi-web --no-docker
233
+ ufo ship hi-web-prod--no-docker
237
234
  ```
238
235
  Note if you use the `--no-docker` option you should ensure that you have already push a docker image to your docker register. Or else the task will not be able to spin up because the docker image does not exist. I recommend that you normally use `ufo ship` most of the time.
239
236
 
240
237
 
241
- ## Automated Docker Images Clean Up
238
+ ### Automated Docker Images Clean Up
242
239
 
243
240
  Ufo can be configured to automatically clean old images from the ECR registry after the deploy completes. I normally set `~/.ufo/settings.yml` like so:
244
241
 
245
242
  ```yaml
246
- ecr_keep: 3
243
+ ecr_keep: 30
247
244
  ```
248
245
 
249
246
  Automated Docker images clean up only works if you are using ECR registry.
250
247
 
248
+ ## Running a single task
249
+
250
+ Sometimes you do not want to run a long running `service` but a one time task. Running Rails migrations are a good example of a one off task. Here is an example of how you would run a one time task.
251
+
252
+ ```
253
+ ufo task hi-migrate-prod
254
+ ```
255
+
256
+ You will need to define a task definition for the migrate command also in `ufo/task_definitions.rb`. If you only need to override the Docker command and can re-use an existing task definition like `hi-web-prod`. You can use the `--override-command` option:
257
+
258
+ ```
259
+ ufo task hi-web-prod --override-command '["bin/migrate"]'
260
+ ufo task hi-web-prod --override-command bin/migrate
261
+ ufo task hi-web-prod --override-command "bin/with_env bundle exec rake db:migrate:redo VERSION=xxx"
262
+ ufo task hi-web-prod --override-command '["bin/with_env", "bundle", "exec", "rake", "db:migrate:redo", "VERSION=xxx"]''
263
+ ```
264
+
265
+ The `--override-command` option takes a string. If the string has brackets in it then it will be evaluated as an Array but the option must be a string.
266
+
251
267
  ## Scale
252
268
 
253
269
  There is a convenience wrapper that simple executes `aws ecs update-service --service [SERVICE] --desired-count [COUNT]`
@@ -13,6 +13,11 @@ common = {
13
13
  cpu: 128,
14
14
  memory_reservation: 256,
15
15
  environment: helper.env_file(".env")
16
+ # another example
17
+ # environment: helper.env_vars(%Q{
18
+ # RAILS_ENV=production
19
+ # SECRET_KEY_BASE=secret
20
+ # })
16
21
  }
17
22
 
18
23
  task_definition "<%= @app %>-web" do
data/lib/ufo.rb CHANGED
@@ -17,6 +17,7 @@ module Ufo
17
17
  autoload :TasksBuilder, 'ufo/tasks_builder'
18
18
  autoload :TasksRegister, 'ufo/tasks_register'
19
19
  autoload :Ship, 'ufo/ship'
20
+ autoload :Task, 'ufo/task'
20
21
  autoload :EcrCleaner, 'ufo/ecr_cleaner'
21
22
  autoload :Destroy, 'ufo/destroy'
22
23
  autoload :Scale, 'ufo/scale'
@@ -95,23 +95,12 @@ module Ufo
95
95
  option :ecr_keep, type: :numeric, desc: "ECR specific cleanup of old images. Specifies how many images to keep. Only runs if the images are ECR images. Defaults to keeping all the images."
96
96
  long_desc Help.ship
97
97
  def ship(service)
98
- builder = DockerBuilder.new(options) # outside if because it need docker.full_image_name
99
- if options[:docker]
100
- builder.build
101
- builder.push
102
- end
103
-
104
- # task definition and deploy logic are coupled in the Ship class.
105
- # Example: We need to know if the task defintion is a web service to see if we need to
106
- # add the elb target group. The web service information is in the TasksBuilder
107
- # and the elb target group gets set in the Ship class.
108
- # So we always call these together.
109
- TasksBuilder.new(options).build
98
+ builder = build_docker(options)
110
99
  task_definition = options[:task] || service # convention
111
- TasksRegister.register(task_definition, options)
112
- ship = Ship.new(service, task_definition, options)
100
+ register_task(task_definition, options)
101
+ return if ENV['TEST'] # allows quick testing of the ship CLI portion only
113
102
 
114
- return if ENV['TEST'] # to allow me to quickly test most of the ship CLI portion only
103
+ ship = Ship.new(service, task_definition, options)
115
104
  ship.deploy
116
105
  if options[:docker]
117
106
  DockerCleaner.new(builder.image_name, options).cleanup
@@ -120,6 +109,16 @@ module Ufo
120
109
  puts "Docker image shipped: #{builder.full_image_name.green}"
121
110
  end
122
111
 
112
+ desc "task [TASK_DEFINITION]", "runs a one time task"
113
+ long_desc Help.task
114
+ option :docker, type: :boolean, desc: "Enable docker build and push", default: true
115
+ option :command, desc: "Override the command used for the container"
116
+ def task(task_definition)
117
+ build_docker(options)
118
+ register_task(task_definition, options)
119
+ Task.new(task_definition, options).run
120
+ end
121
+
123
122
  desc "destroy [SERVICE]", "destroys the ECS service"
124
123
  long_desc Help.destroy
125
124
  option :force, type: :boolean, desc: "By pass are you sure prompt."
@@ -133,5 +132,31 @@ module Ufo
133
132
  def scale(service, count)
134
133
  Scale.new(service, count, options).update
135
134
  end
135
+
136
+ desc "version", "Prints version number of installed ufo"
137
+ def version
138
+ puts Ufo::VERSION
139
+ end
140
+
141
+ no_tasks do
142
+ def build_docker(options)
143
+ builder = DockerBuilder.new(options) # outside if because it need docker.full_image_name
144
+ if options[:docker]
145
+ builder.build
146
+ builder.push
147
+ end
148
+ builder
149
+ end
150
+
151
+ def register_task(task_definition, options)
152
+ # task definition and deploy logic are coupled in the Ship class.
153
+ # Example: We need to know if the task defintion is a web service to see if we need to
154
+ # add the elb target group. The web service information is in the TasksBuilder
155
+ # and the elb target group gets set in the Ship class.
156
+ # So we always call these together.
157
+ TasksBuilder.new(options).build
158
+ TasksRegister.register(task_definition, options)
159
+ end
160
+ end
136
161
  end
137
162
  end
@@ -137,6 +137,28 @@ $ ufo ship hi-web-prod --no-elb-prompt
137
137
  EOL
138
138
  end
139
139
 
140
+ def task
141
+ <<-EOL
142
+ Examples:
143
+
144
+ To run a one time task with ECS:
145
+
146
+ $ ufo task hi-migrate-prod
147
+
148
+ You can also override the command used by the Docker container in the task definitions via override-command.
149
+
150
+ ufo task hi-web-prod --override-command bin/migrate
151
+
152
+ ufo task hi-web-prod --override-command '["bin/migrate"]'
153
+
154
+ ufo task hi-web-prod --override-command "bin/with_env bundle exec rake db:migrate:redo VERSION=xxx"
155
+
156
+ ufo task hi-web-prod --override-command '["bin/with_env","bundle","exec","rake","db:migrate:redo","VERSION=xxx"]''
157
+
158
+ The `--override-command` option takes a string. If the string has brackets in it then it will be evaluated as an Array but the optoin must be a string. Also there can be no spaces in the string.
159
+ EOL
160
+ end
161
+
140
162
  def destroy
141
163
  <<-EOL
142
164
  Examples:
@@ -15,7 +15,7 @@ module Ufo
15
15
  @data = YAML.load_file(settings_path)
16
16
  @data = user_settings.merge(@data)
17
17
  else
18
- puts "ERROR: No settings file file at #{settings_path}"
18
+ puts "ERROR: No settings file at #{settings_path}"
19
19
  puts "Please create a settings file via: ufo init"
20
20
  exit 1
21
21
  end
@@ -75,23 +75,6 @@ module Ufo
75
75
  stop_old_task(deployed_service) if @stop_old_tasks
76
76
  end
77
77
 
78
- def process_multiple_services
79
- puts "Multi services mode" unless @options[:mute]
80
- services_to_deploy = []
81
- find_all_ecs_services do |ecs_service|
82
- if service_pattern_match?(ecs_service.service_name)
83
- services_to_deploy << ecs_service
84
- end
85
- end
86
-
87
- deployed_services = services_to_deploy.map do |ecs_service|
88
- update_service(ecs_service)
89
- end
90
-
91
- wait_for_all_deployments(deployed_services) if @wait_for_deployment && !@options[:noop]
92
- stop_old_tasks(deployed_services) if @stop_old_tasks
93
- end
94
-
95
78
  def service_tasks(cluster, service)
96
79
  all_task_arns = ecs.list_tasks(cluster: cluster, service_name: service).task_arns
97
80
  return [] if all_task_arns.empty?
@@ -367,19 +350,6 @@ module Ufo
367
350
  }
368
351
  end
369
352
 
370
- def service_exact_match?(service_name)
371
- service_name == @service
372
- end
373
-
374
- # @service is changed to a pattern to be search with.
375
- #
376
- # Examples:
377
- # @service == "hi-.*-prod"
378
- def service_pattern_match?(service_name)
379
- service_patttern = Regexp.new(@service)
380
- service_name =~ service_patttern
381
- end
382
-
383
353
  def find_ecs_service
384
354
  find_all_ecs_services.find { |ecs_service| ecs_service.service_name == @service }
385
355
  end
@@ -0,0 +1,53 @@
1
+ module Ufo
2
+ class Task
3
+ include Defaults
4
+ include AwsServices
5
+
6
+ def initialize(task_definition, options)
7
+ @task_definition = task_definition
8
+ @options = options
9
+ @cluster = @options[:cluster] || default_cluster
10
+ end
11
+
12
+ def run
13
+ puts "Running task_definition: #{@task_definition}".colorize(:green) unless @options[:mute]
14
+ return if @options[:noop]
15
+ task_options = {
16
+ cluster: @cluster,
17
+ task_definition: @task_definition
18
+ }
19
+ task_options.merge!(overrides: overrides) if @options[:command]
20
+ resp = ecs.run_task(task_options)
21
+ puts "Task ARN: #{resp.tasks[0].task_arn}" unless @options[:mute]
22
+ end
23
+
24
+ private
25
+ # only using the overrides to override the container command
26
+ def overrides
27
+ command = @options[:command]
28
+ command = eval(command) if command.include?('[') # command is in Array form
29
+ container_definition = get_original_container_definition
30
+ {
31
+ container_overrides: [
32
+ {
33
+ name: container_definition[:name],
34
+ command: command,
35
+ environment: container_definition[:environment],
36
+ },
37
+ ]
38
+ }
39
+ end
40
+
41
+ def get_original_container_definition
42
+ resp = ecs.list_task_definitions(
43
+ family_prefix: @task_definition,
44
+ sort: "DESC"
45
+ )
46
+ # "arn:aws:ecs:us-east-1:<aws_account_id>:task-definition/wordpress:6",
47
+ last_definition_arn = resp.task_definition_arns.first
48
+ task_name = last_definition_arn.split("/").last
49
+ resp = ecs.describe_task_definition(task_definition: task_name)
50
+ container_definition = resp.task_definition.container_definitions[0].to_h
51
+ end
52
+ end
53
+ end
@@ -1,3 +1,3 @@
1
1
  module Ufo
2
- VERSION = "0.1.6"
2
+ VERSION = "1.0.0"
3
3
  end
@@ -46,5 +46,12 @@ describe Ufo::CLI do
46
46
  expect(out).to include("Task Definitions built")
47
47
  end
48
48
  end
49
+
50
+ context "task" do
51
+ it "runs one time task" do
52
+ out = execute("bin/ufo task hi-migrate-prod #{@args}")
53
+ expect(out).to include("Running task_definition")
54
+ end
55
+ end
49
56
  end
50
57
  end
@@ -3,7 +3,13 @@ require 'spec_helper'
3
3
  describe Ufo::EcrCleaner do
4
4
  let(:docker_image_name) { "123456789.dkr.ecr.us-east-1.amazonaws.com/my-name" }
5
5
  let(:repo_domain) { "https://123456789.dkr.ecr.us-east-1.amazonaws.com" }
6
- let(:cleaner) { Ufo::EcrCleaner.new(docker_image_name, mute: true) }
6
+ let(:cleaner) do
7
+ Ufo::EcrCleaner.new(docker_image_name,
8
+ project_root: "spec/fixtures/hi",
9
+ ecr_keep: 3, # using 3 to test, default is 30
10
+ mute: true
11
+ )
12
+ end
7
13
  before(:each) do
8
14
  allow(cleaner).to receive(:update_auth_token).and_return(:whatever)
9
15
  end
@@ -11,14 +11,20 @@ describe Ufo::Ship do
11
11
  stop_old_tasks: false,
12
12
  }
13
13
  end
14
+ let(:service) { "hi-web-prod" }
14
15
  let(:task_definition) { service }
15
- let(:ship) { Ufo::Ship.new(service, task_definition, options) }
16
+ let(:ship) do
17
+ ship = Ufo::Ship.new(service, task_definition, options)
18
+ allow(ship).to receive(:ecs).and_return(ecs_client)
19
+ ship
20
+ end
16
21
 
17
- context "single service id" do
18
- let(:service) { "hi-web-prod" }
19
- it "should create or update single service" do
22
+ context "hi-web-prod service" do
23
+ it "should create or update service" do
20
24
  allow(ship).to receive(:process_single_service)
25
+
21
26
  ship.deploy
27
+
22
28
  expect(ship).to have_received(:process_single_service)
23
29
  end
24
30
 
@@ -26,49 +32,37 @@ describe Ufo::Ship do
26
32
  it "should create service on first cluster" do
27
33
  allow(ship).to receive(:find_ecs_service)
28
34
  allow(ship).to receive(:create_service)
35
+
29
36
  ship.deploy
37
+
30
38
  expect(ship).to have_received(:create_service)
31
39
  end
32
40
  end
33
41
 
34
42
  context "1 services found" do
35
- it "should call update service 3 times" do
43
+ it "should call update service" do
36
44
  allow(ship).to receive(:find_ecs_service).and_return(ecs_service("hi-web-prod"))
37
45
  allow(ship).to receive(:update_service)
46
+
38
47
  ship.deploy
48
+
39
49
  expect(ship).to have_received(:update_service).exactly(1).times
40
50
  end
41
51
  end
42
52
  end
43
53
 
44
- context "multiple service pattern" do
45
- let(:service) { "hi-.*-prod" }
46
- it "should update multiple service that are found" do
47
- allow(ship).to receive(:process_multiple_services)
48
- ship.deploy
49
- expect(ship).to have_received(:process_multiple_services)
50
- end
51
-
52
- context "0 services found" do
53
- it "should not call update_service" do
54
- allow(ship).to receive(:find_all_ecs_services)
55
- allow(ship).to receive(:update_service)
56
- ship.deploy
57
- expect(ship).to_not have_received(:update_service)
58
- end
59
- end
54
+ # mocks
55
+ def ecs_client
56
+ ecs = double("ecs")
57
+ allow(ecs).to receive(:describe_clusters).and_return(ecs_describe_clusters)
58
+ ecs
59
+ end
60
60
 
61
- context "3 services found" do
62
- it "should call update service 3 times" do
63
- allow(ship).to receive(:find_all_ecs_services).
64
- and_yield(ecs_service("hi-web-prod")).
65
- and_yield(ecs_service("hi-worker-prod")).
66
- and_yield(ecs_service("hi-clock-prod"))
67
- allow(ship).to receive(:update_service)
68
- ship.deploy
69
- expect(ship).to have_received(:update_service).exactly(3).times
70
- end
71
- end
61
+ # ensure_cluster_exist calls this and this makes sure that the cluster 'exists'
62
+ def ecs_describe_clusters
63
+ describe_clusters = double("ecs-describe-clusters")
64
+ allow(describe_clusters).to receive(:clusters).and_return(["cluster1"])
65
+ describe_clusters
72
66
  end
73
67
 
74
68
  def ecs_service(name)
@@ -0,0 +1,33 @@
1
+ require 'spec_helper'
2
+
3
+ describe Ufo::Ship do
4
+ let(:project_root) { File.expand_path("../../fixtures/hi", __FILE__) }
5
+ let(:options) do
6
+ {
7
+ project_root: project_root,
8
+ mute: true
9
+ }
10
+ end
11
+ let(:task_definition) { "hi-migrate-prod" }
12
+ let(:task) do
13
+ task = Ufo::Task.new(task_definition, options)
14
+ allow(task).to receive(:ecs).and_return(ecs_client)
15
+ task
16
+ end
17
+
18
+ context "hi-migrate-prod" do
19
+ it "should migrate the database" do
20
+ task.run
21
+
22
+ expect(task.ecs).to have_received(:run_task)
23
+ end
24
+ end
25
+
26
+ # mocks
27
+ def ecs_client
28
+ ecs = double("ecs")
29
+ fake_response = double('fake-response').as_null_object
30
+ allow(ecs).to receive(:run_task).and_return(fake_response)
31
+ ecs
32
+ end
33
+ end
@@ -11,7 +11,7 @@ require "#{root}/lib/ufo"
11
11
 
12
12
  module Helpers
13
13
  def execute(cmd)
14
- puts "Running: #{cmd}" if ENV['DEBUG']
14
+ puts "Running: #{cmd.colorize(:magenta)}" if ENV['DEBUG']
15
15
  out = `#{cmd}`
16
16
  puts out if ENV['DEBUG']
17
17
  out
@@ -25,4 +25,9 @@ end
25
25
 
26
26
  RSpec.configure do |c|
27
27
  c.include Helpers
28
+
29
+ c.before(:each) do
30
+ # ensures we never called real aws api since the fixture home folder does not have ~/.aws/credentails setup
31
+ ENV['HOME'] = "spec/fixtures/home"
32
+ end
28
33
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: ufo
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.6
4
+ version: 1.0.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Tung Nguyen
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2017-04-28 00:00:00.000000000 Z
11
+ date: 2017-05-08 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: thor
@@ -223,6 +223,7 @@ files:
223
223
  - lib/ufo/scale.rb
224
224
  - lib/ufo/settings.rb
225
225
  - lib/ufo/ship.rb
226
+ - lib/ufo/task.rb
226
227
  - lib/ufo/tasks_builder.rb
227
228
  - lib/ufo/tasks_register.rb
228
229
  - lib/ufo/templates/default.json.erb
@@ -232,6 +233,7 @@ files:
232
233
  - spec/lib/ecr_auth_spec.rb
233
234
  - spec/lib/ecr_cleaner_spec.rb
234
235
  - spec/lib/ship_spec.rb
236
+ - spec/lib/task_spec.rb
235
237
  - spec/spec_helper.rb
236
238
  - ufo.gemspec
237
239
  homepage: https://github.com/tongueroo/ufo
@@ -254,7 +256,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
254
256
  version: '0'
255
257
  requirements: []
256
258
  rubyforge_project:
257
- rubygems_version: 2.6.8
259
+ rubygems_version: 2.5.2
258
260
  signing_key:
259
261
  specification_version: 4
260
262
  summary: Build Docker Containers and Ship Them to AWS ECS
@@ -264,4 +266,5 @@ test_files:
264
266
  - spec/lib/ecr_auth_spec.rb
265
267
  - spec/lib/ecr_cleaner_spec.rb
266
268
  - spec/lib/ship_spec.rb
269
+ - spec/lib/task_spec.rb
267
270
  - spec/spec_helper.rb