ufo 1.5.0 → 1.6.0

data/docs/_docs/ufo-scale.md

@@ -0,0 +1,22 @@
+---
+title: ufo scale
+---
+
+Ufo provides a command to quickly scale up and down an ECS service. Here's an example of how you use it:
+
+```sh
+ufo scale hi-web-stag 3
+```
+
+You should get output similiar to below:
+
+```sh
+Scale hi-web-stag service in stag cluster to 3
+```
+
+While scaling via this method is quick and convenient the [ECS Service AutoScaling](http://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-auto-scaling.html) that is built into ECS is a much more powerful way to manage scaling your ECS service.
+
+<a id="prev" class="btn btn-basic" href="{% link _docs/ufo-ships.md %}">Back</a>
+<a id="next" class="btn btn-primary" href="{% link _docs/ufo-destroy.md %}">Next Step</a>
+<p class="keyboard-tip">Pro tip: Use the <- and -> arrow keys to move back and forward.</p>
+

data/docs/_docs/ufo-settings.md

@@ -0,0 +1,57 @@
+---
+title: Settings
+---
+
+The behavior of ufo can be configured via the `ufo/settings.yml` file.  A starter `settings.yml` file is generated as part of the `ufo init` command. A lot of these settings are default options so you do not have to type out these options repeatedly for some of the ufo commands. Let's cover the available settings in an example `settings.yml` that we've modified:
+
+```yaml
+image: tongueroo/hi
+clean_keep: 30
+ecr_keep: 30
+service_cluster:
+  default: stag # default cluster
+  hi-web-stag: stag
+  hi-clock-stag: blue
+  hi-worker-stag: anothercluster
+```
+
+### Docker image name
+
+The `image` value is the name that ufo will use for the Docker image name to be built.  Only provide the basename part of the image name without the tag. For example, `tongueroo/hi` is correct and `tongueroo/hi:my-tag` is incorrect.
+
+### ECS Cluster for Service
+
+The `service_cluster` mapping provides a way to set default "service-to-cluster" mappings so that you do not have to specify the `--cluster` repeatedly.  This is very helpful. For example:
+
+```sh
+ufo ship hi-web-stag --cluster stag
+ufo ship hi-web-stag # same as above because it is configured in ufo/settings.yml
+ufo ship hi-web-stag --cluster special-cluster # overrides the default setting in `ufo/settings.yml`
+```
+
+Also, with this `settings.yml`:
+
+```sh
+ufo ship hi-clock-stag # deploys to the ECS cluster named blue
+```
+
+### ECR Cleanup
+
+Ufo can be configured to automatically clean old images from the ECR registry after the deploy completes.
+
+```yaml
+ecr_keep: 30
+```
+
+### Docker Cleanup
+
+Ufo can be configured to automatically clean old Docker images.
+
+```yaml
+docker_keep: 30
+```
+
+<a id="prev" class="btn btn-basic" href="{% link _docs/ufo-help.md %}">Back</a>
+<a id="next" class="btn btn-primary" href="{% link _docs/conventions.md %}">Next Step</a>
+<p class="keyboard-tip">Pro tip: Use the <- and -> arrow keys to move back and forward.</p>
+

data/docs/_docs/ufo-ship.md

@@ -0,0 +1,75 @@
+---
+title: ufo ship
+---
+
+The main command you use when using ufo is: `ufo ship`.  This command builds the docker image, registers the generated ECS task definition and deploys the definition to AWS ECS all in one go.
+
+The command has a decent amount of options, you can see the options available with `ufo ship -h`.  The table below covers some of the options in detail:
+
+{% include ufo-ship-options.md %}
+
+As you can see there are plenty of options for `ufo ship`.  Let's demonstrate usage of them in a few examples:
+
+#### Load Balancer Target Group
+
+When you are deploying to a service with the word 'web' in it, ufo by convention assumes that this is a web service that uses a load balancer in front of it.  This is also covered a in the [Conventions]({% link _docs/conventions.md %}) page.  If you would you like to create a service with the word web without an load balancer associated with it you can use the `--no-target-group-prompt` option:
+
+```sh
+ufo ship hi-web-stag --no-target-group-prompt
+```
+
+Or if you would like specify the target-group up front and not be bother with the prompted later you can use the `--target-group` option.
+
+```sh
+ufo ship hi-web-stag --target-group=arn:aws:elasticloadbalancing:us-east-1:12345689:targetgroup/hi-web-prod/12345
+```
+
+#### Deploying Existing Task Definition
+
+Let's say you already have built an registered a task definition by some other means and only want to use ufo to deploy that already registered task definition. You can do this by skipping the task build and register phase. It probably also makes sense to skip the docker phase in this case.
+
+```sh
+ufo ship hi-web-stag --no-docker --no-tasks
+```
+
+#### Waiting for Deployments to Complete
+
+By default when ufo updates the ECS service with the new task definition it does so asynchronuously. You then normally visit the ECS service console and then refresh until you see that the deployment is completed.  You can also have ufo poll and wait for the deployment to be done with the `--wait` option
+
+```sh
+ufo ship hi-web-stag --wait
+```
+
+You should see output similar to this:
+
+```sh
+Shipping hi-web-stag...
+hi-web-stag service updated on stag cluster with task hi-web-stag
+Waiting for deployment of task definition hi-web-stag:8 to complete
+......
+Time waiting for ECS deployment: 31s.
+Software shipped!
+```
+
+#### Cleaning up Docker Images Automatically
+
+Since ufo builds the Docker image every time there's a deployment you will end up with a long list of docker images.  Ufo automatically cleans up older docker images at the end of the deploy process if you are using AWS ECR.  By default ufo keeps the most recent 30 Docker images. This can be adjust with the `--ecr-keep` option.
+
+```sh
+docker ship hi-web-stag --ecr-keep 2
+```
+
+You should see something like this:
+
+```sh
+Cleaning up docker images...
+Running: docker rmi tongueroo/hi:ufo-2017-06-12T06-46-12-a18aa30
+```
+
+If you are using DockerHub or another registry, ufo does not automatically clean up images.
+
+
+<a id="prev" class="btn btn-basic" href="{% link _docs/ufo-init.md %}">Back</a>
+<a id="next" class="btn btn-primary" href="{% link _docs/ufo-ships.md %}">Next Step</a>
+<p class="keyboard-tip">Pro tip: Use the <- and -> arrow keys to move back and forward.</p>
+

data/docs/_docs/ufo-ships.md

@@ -0,0 +1,52 @@
+---
+title: ufo ships
+---
+
+The `ufo ships` command allows you to deploy the *same* Docker image and task definition to multiple ECS services.  This is a very common pattern to have the same code base running on different roles.  For example, say you have an app with 3 roles:
+
+1. web - serves web requests.
+2. worker - processes background jobs.
+3. clock - schedules background jobs.
+
+Instead of using the [ufo ship]({% link _docs/ufo-ship.md %}) and build and deploying the code 3 times you can instead use `ufo ships`.  This will result in the *same* Docker image and *same* task definition being deployed to all 3 services.  Example usage:
+
+```sh
+ufo ships hi-web-stag hi-worker-stag hi-clock-stag
+```
+
+### Shell expansion
+
+Since the ECS service names are provided as a list you can shorten the command by using bash shell expansion 😁
+
+```sh
+ufo ships hi-{web,worker,clock}-stag
+```
+
+If you're new to shell expansion, run this to understand why above works just as well:
+
+```sh
+$ echo hi-{web,worker,clock}-stag
+hi-web-stag hi-worker-stag hi-clock-stag
+```
+
+### Overriding convention
+
+As explained in detail in [Conventions]({% link _docs/conventions.md %}) the task definition and service name are the same by convention.  This convention also applies for each of the services being shipped in the list. The task definition and service names match for each of the services in the list.  If you would like to override the convention as part of the ships command then you use a special syntax. In the special syntax the service and task definition is separated by a colon.  Examples:
+
+```sh
+ufo ships hi-web-prod-1:hi-web-prod hi-clock-prod-1 hi-worker-prod-1
+ufo ships hi-web-prod-1:my-task hi-clock-prod-1:another-task hi-worker-prod-1:third-task
+```
+
+### ufo ships Options
+
+The `ufo ships` and `ufo ship` command support the same options. The options are presented here again for convenience:
+
+{% include ufo-ship-options.md %}
+
+Note: The `--task` option is not used with the `ufo ships` command.
+
+<a id="prev" class="btn btn-basic" href="{% link _docs/ufo-ship.md %}">Back</a>
+<a id="next" class="btn btn-primary" href="{% link _docs/ufo-scale.md %}">Next Step</a>
+<p class="keyboard-tip">Pro tip: Use the <- and -> arrow keys to move back and forward.</p>
+

data/docs/_docs/ufo-tasks-build.md

@@ -0,0 +1,150 @@
+---
+title: ufo tasks build
+---
+
+The command `ufo tasks build` generates the task definitions locally and writes them to the `output/` folder.  There are 2 files that it uses in order to produce the raw AWS task definitions files.
+
+1. ufo/templates/main.json.erb
+2. ufo/task_definitions.rb
+
+Here's an example of each of them:
+
+**main.json.erb**:
+
+```json
+{
+    "family": "<%= @family %>",
+    "containerDefinitions": [
+        {
+            "name": "<%= @name %>",
+            "image": "<%= @image %>",
+            "cpu": <%= @cpu %>,
+            <% if @memory %>
+            "memory": <%= @memory %>,
+            <% end %>
+            <% if @memory_reservation %>
+            "memoryReservation": <%= @memory_reservation %>,
+            <% end %>
+            <% if @container_port %>
+            "portMappings": [
+                {
+                    "containerPort": "<%= @container_port %>",
+                    "protocol": "tcp"
+                }
+            ],
+            <% end %>
+            "command": <%= @command.to_json %>,
+            <% if @environment %>
+            "environment": <%= @environment.to_json %>,
+            <% end %>
+            <% if @awslogs_group %>
+            "logConfiguration": {
+                "logDriver": "awslogs",
+                "options": {
+                    "awslogs-group": "<%= @awslogs_group %>",
+                    "awslogs-region": "<%= @awslogs_region || 'us-east-1' %>",
+                    "awslogs-stream-prefix": "<%= @awslogs_stream_prefix %>"
+                }
+            },
+            <% end %>
+            "essential": true
+        }
+    ]
+}
+```
+
+**task_definitions.rb**:
+
+```
+# common variables
+common = {
+  image: helper.full_image_name, # includes the git sha tongueroo/hi:ufo-[sha].
+  cpu: 128,
+  memory_reservation: 256,
+  environment: helper.env_file(".env")
+  # another example
+  # environment: helper.env_vars(%Q{
+  #   RAILS_ENV=production
+  #   SECRET_KEY_BASE=secret
+  # })
+}
+
+task_definition "hi-web-stag" do
+  source "main" # will use ufo/templates/main.json.erb
+  variables(common.dup.deep_merge(
+    family: task_definition_name,
+    name: "web",
+    container_port: helper.dockerfile_port,
+    command: ["bin/web"]
+  ))
+end
+
+task_definition "hi-worker-stag" do
+  source "main" # will use ufo/templates/main.json.erb
+  variables(common.dup.deep_merge(
+    family: task_definition_name,
+    name: "worker",
+    command: ["bin/worker"]
+  ))
+end
+
+task_definition "hi-clock-stag" do
+  source "main" # will use ufo/templates/main.json.erb
+  variables(common.dup.deep_merge(
+    family: task_definition_name,
+    name: "clock",
+    command: ["bin/clock"]
+  ))
+end
+```
+
+Ufo uses the ERB template in `main.json.erb` and combines it with the variables defined in the task definition declarations in `task_definitions.rb` and generates the raw AWS formatted task definition in the `output` folder.  In this case there are 3 task_definitions so there will be 3 generated output files.
+
+To build the task definitions:
+
+```sh
+ufo tasks build
+```
+
+You should see output similar to below:
+
+<img src="/img/tutorials/ufo-tasks-build.png" class="doc-photo" />
+
+Let's take a look at one of the generated files: `ufo/output/hi-web-stag.json`.
+
+```json
+{
+  "family": "hi-web-stag",
+  "containerDefinitions": [
+    {
+      "name": "web",
+      "image": "tongueroo/hi:ufo-2017-06-11T22-22-32-a18aa30",
+      "cpu": 128,
+      "memoryReservation": 256,
+      "portMappings": [
+        {
+          "containerPort": "3000",
+          "protocol": "tcp"
+        }
+      ],
+      "command": [
+        "bin/web"
+      ],
+      "environment": [
+        {
+          "name": "RAILS_ENV",
+          "value": "staging"
+        }
+      ],
+      "essential": true
+    }
+  ]
+}
+```
+
+If you need to modify the task definition template to suite your own needs it is super simple, just edit `main.json.erb`.  No need to dive deep into internal code that builds up the task definition with some internal structure.  It is all there for you to fully control.
+
+<a id="prev" class="btn btn-basic" href="{% link _docs/ufo-docker-clean.md %}">Back</a>
+<a id="next" class="btn btn-primary" href="{% link _docs/ufo-tasks-register.md %}">Next Step</a>
+<p class="keyboard-tip">Pro tip: Use the <- and -> arrow keys to move back and forward.</p>
+

data/docs/_docs/ufo-tasks-register.md

@@ -0,0 +1,26 @@
+---
+title: ufo tasks register
+---
+
+The `ufo tasks register` command registers all the generated task definitions in `output/` to AWS ECS. Let's run it:
+
+```sh
+ufo ecs register
+```
+
+You should see something similiar to this:
+
+```sh
+hi-clock-stag task definition registered.
+hi-web-stag task definition registered.
+hi-worker-stag task definition registered.
+```
+
+You can verify that the task definitions have been registered properly by viewing the AWS ECS Console Task Definitions page.  You should see something similar to this:
+
+<img src="/img/tutorials/ecs-console-task-definitions.png" class="doc-photo" />
+
+<a id="prev" class="btn btn-basic" href="{% link _docs/ufo-tasks-build.md %}">Back</a>
+<a id="next" class="btn btn-primary" href="{% link _docs/ufo-help.md %}">Next Step</a>
+<p class="keyboard-tip">Pro tip: Use the <- and -> arrow keys to move back and forward.</p>
+

data/docs/_includes/about.html

@@ -0,0 +1,19 @@
+ <!-- About Section -->
+    <section class="success" id="about">
+        <div class="container">
+            <div class="row">
+                <div class="col-lg-12 text-center">
+                    <h2>About</h2>
+                    <hr class="star-light">
+                </div>
+            </div>
+            <div class="row">
+                <div class="col-lg-4 col-lg-offset-2">
+                    <p>Ufo is a tool that makes building and shipping Docker containers to AWS ECS super easy. At a basic level, ufo builds the docker container image, registers that image to ECS as a task definition, and then deploys that task definition to ECS by updating the service. Ufo provides a `ufo ship` command that deploys your code to ECS.</p>
+                </div>
+                <div class="col-lg-4">
+                    <p>Ufo was built directly from real life production use cases after seeing the same patterns repeated over and over. Ufo allows you to write the AWS task definition json format file in ERB, an easy templating language.  This allows you to reuse the ufo tool with multiple applications and only put the truly application specific business logic in each app code base.</p>
+                </div>
+            </div>
+        </div>
+    </section>

data/docs/_includes/commands.html

@@ -0,0 +1,78 @@
+ <!-- About Section -->
+    <section id="about">
+        <div class="container">
+            <div class="row">
+                <div class="col-lg-12 text-center">
+                    <h2>Commands</h2>
+                    <hr class="star-primary">
+                </div>
+            </div>
+            <div class="row">
+                <div class="col-lg-8 col-lg-offset-2">
+                    <div class="commands">
+{% highlight sh %}
+ufo ship
+
+ufo docker build
+
+ufo tasks build
+
+ufo scale
+
+ufo destroy
+{% endhighlight %}
+                    </div>
+                </div>
+            </div>
+            <div class="row">
+                <div class="col-lg-6">
+                    <h3>ufo/templates/main.json.erb</h3>
+                    <div class="commands">
+{% highlight json %}
+{
+    "family": "<%= @family %>",
+    "containerDefinitions": [
+        {
+            "name": "<%= @name %>",
+            "image": "<%= @image %>",
+            "cpu": <%= @cpu %>,
+            <% end %>
+            "command": <%= @command.to_json %>,
+            <% if @environment %>
+            "environment": <%= @environment.to_json %>,
+            <% end %>
+            "essential": true
+        }
+    ]
+}
+{% endhighlight %}
+                    </div>
+                </div>
+                <div class="col-lg-6">
+                    <h3>ufo/template_definitions.rb</h3>
+                    <div class="commands">
+{% highlight ruby %}
+common = {
+  image: helper.full_image_name,
+  cpu: 128,
+  memory_reservation: 256,
+  environment: helper.env_vars("RAILS_ENV=production"),
+  container_port: helper.dockerfile_port,
+}
+
+task_definition "example-web" do
+  source "main"
+  variables(common.dup.deep_merge(
+    family: task_definition_name,
+    name: "web",
+    awslogs_group: "example-web",
+    awslogs_stream_prefix: "example",
+    command: ["bin/web"]
+  ))
+end
+{% endhighlight %}
+                    </div>
+                </div>
+            </div>
+        </div>
+    </section>