ufo 2.3.0 → 3.0.0

data/docs/_docs/structure.md

@@ -2,10 +2,10 @@
 title: Structure
 ---
 
-Ufo creates a ufo folder within your project which contains the required files used by ufo to build and deploy docker images to ECS.  The standard directory structure of the ufo folder looks like this:
+Ufo creates a `.ufo` folder within your project which contains the required files used by ufo to build and deploy docker images to ECS.  The standard directory structure of the `.ufo` folder looks like this:
 
 ```sh
-ufo
+.ufo
 ├── output
 ├── settings.yml
 ├── task_definitions.rb

data/docs/_docs/tutorial-ufo-docker-build.md

@@ -48,19 +48,26 @@ $
 
 As you can see `ufo docker build` effectively shells out and calls `docker build -t tongueroo/hi:ufo-2017-06-11T22-18-03-a18aa30 -f Dockerfile .`.  The docker image tag that is generated contains a useful timestamp and the current HEAD git sha of the project that you are on.
 
-By default when you are running `ufo docker build` directly it does not push the docker image to the registry.  If you would like it to automaticaly push the built image to a registry at the end of the build use the `--push` flag.
+By default when you are running `ufo docker build` directly it does not push the docker image to the registry.  If you would like it to push the built image to a registry at the end of the build use the `--push` flag.
 
 ```sh
 ufo docker build --push
 ```
 
-You should see it being pushed at the end:
+You can also use the `ufo docker push` command which will push the last built image from `ufo docker build`.
+
+```
+ufo docker push
+```
+
+You should see the image being pushed with a message that looks something like this:
 
 ```sh
 Pushed tongueroo/hi:ufo-2017-06-11T22-22-32-a18aa30 docker image. Took 9s.
 ```
 
-Note in order to push the image to a register you will need to login into the registry.  If you are using DockerHub use the `docker login` command.  If you are using AWS ECR then you can use the `aws ecr get-login` command.
+
+Note in order to push the image to a registry you will need to login into the registry.  If you are using DockerHub use the `docker login` command.  If you are using AWS ECR then you can use the `aws ecr get-login` command.
 
 <a id="prev" class="btn btn-basic" href="{% link _docs/tutorial-ufo-init.md %}">Back</a>
 <a id="next" class="btn btn-primary" href="{% link _docs/tutorial-ufo-tasks-build.md %}">Next Step</a>

data/docs/_docs/tutorial-ufo-init.md

@@ -13,18 +13,19 @@ ufo init --app=hi --image=tongueroo/hi
 
 You should see output similiar to this:
 
-```
+```sh
 $ ufo init --app=hi --image=tongueroo/hi
 Setting up ufo project...
-created: ./bin/deploy
-created: ./Dockerfile
-created: ./ufo/settings.yml
-created: ./ufo/task_definitions.rb
-created: ./ufo/templates/main.json.erb
-created: ./ufo/variables/base.rb
-created: ./ufo/variables/prod.rb
-created: ./ufo/variables/stag.rb
-created: ./.env
+      create  .env
+      create  .ufo/settings.yml
+      create  .ufo/task_definitions.rb
+      create  .ufo/templates/main.json.erb
+      create  .ufo/variables/base.rb
+      create  .ufo/variables/development.rb
+      create  .ufo/variables/production.rb
+      create  Dockerfile
+      create  bin/deploy
+      append  .gitignore
 Starter ufo files created.
 $ ufo ship hi-web
 Building docker image with:
@@ -37,25 +38,56 @@ $
 The `ufo init` command generated a few starter ufo files for you. The standard directory structure of the ufo folder looks like this:
 
 ```sh
-ufo
+.ufo
 ├── output
 ├── settings.yml
 ├── task_definitions.rb
-└── templates
-    └── main.json.erb
+├── templates
+|   └── main.json.erb
+└── variables
+    ├── base.rb
+    ├── prod.rb
+    └── stag.rb
 ```
 
 The explanation of the folders and files were covered in detailed earlier at [Structure]({% link _docs/structure.md %}).
 
 ### Settings
 
-Take a look at the `ufo/settings.yml` file and notice that it contains some default configuration settings so you do not have to type out these options repeatedly for some of the ufo commands.
+Take a look at the `settings.yml` file and notice that it contains some default configuration settings so you do not have to type out these options repeatedly for some of the ufo commands.
 
 ```yaml
-image: tongueroo/hi
+# More info: http://ufoships.com/docs/ufo-settings/
+base: &base
+  image: tongueroo/hi
+  # clean_keep: 30 # cleans up docker images on your docker server.
+  # ecr_keep: 30 # cleans up images on ECR and keeps this remaining amount. Defaults to keep all.
+  # defaults when an new ECS service is created by ufo ship
+  new_service:
+    maximum_percent: 200
+    minimum_healthy_percent: 100
+    desired_count: 1
+
+development:
+  <<: *base
+  # cluster: dev # uncomment if you want the cluster name be other than the default
+                 # the default is to match UFO_ENV.  So UFO_ENV=development means the ECS
+                 # cluster will be name development
+  # When you have AWS_PROFILE set to one of these values, ufo will switch to the desired
+  # environment. This prevents you from switching AWS_PROFILE, forgetting to
+  # also switch UFO_ENV, and accidentally deploying to production vs development.
+  # aws_profiles:
+  #   - dev_profile1
+  #   - dev_profile2
+
+production:
+  <<: *base
+  # cluster: prod
+  # aws_profiles:
+  #   - prod_profile
 ```
 
-The `image` value is the name that ufo will use as a base to generate a Docker image name.
+The `image` value is the name that ufo will use as a base portion of the name to generate a Docker image name, it should not include the tag portion.
 
 The other settings are optional.  You can learn more about them at [Settings]({% link _docs/settings.md %}).
 

data/docs/_docs/tutorial-ufo-ship.md

@@ -10,10 +10,10 @@ In this guide we have walked through what ufo does step by step.  First ufo buil
 ufo ship hi-web
 ```
 
-By convention, ufo will ship the docker container to an ECS cluster with the same value as UFO_ENV, which defaults to prod.  So the command above is the same as:
+By convention, ufo will ship the docker container to an ECS cluster with the same value as UFO_ENV, which defaults to development.  So the command above is the same as:
 
 ```sh
-ufo ship hi-web --cluster prod
+ufo ship hi-web --cluster production
 UFO_ENV=production ufo ship hi-web --cluster prod
 ```
 
@@ -23,7 +23,9 @@ When you run `ufo ship hi-web`:
 2. Generates a task definition and registers it.
 3. Updates the ECS service to use it.
 
-If the ECS service hi-web does not yet exist, ufo will create the service for you. Ufo will also automatically create the ECS cluster. If you are relying on this tool to create the cluster, you still need to associate ECS Container Instances to the cluster yourself.
+If the ECS service hi-web does not yet exist, ufo will create the service for you. Ufo will also automatically create the ECS cluster.
+
+NOTE: If you are relying on this tool to create the cluster, you still need to associate ECS Container Instances to the cluster yourself.
 
 By convention, if the service has a container name web, you'll get prompted to create an ELB and specify a target group arn.  The ELB and target group must already exist. You can bypass the prompt and specify the target group ARN as part of the ship command or with the `--no-target-group-prompt` option.  The elb target group only gets associated with the ECS service if the service is being created for the first time.  If the service already exists then the `--target-group` parameter just gets ignored and the ECS task simply gets updated.  Example:
 
@@ -66,14 +68,19 @@ Checking the ECS console you should see something like this:
 
 You have successfully shipped a docker image to ECS! 🍾🥂
 
-### Skipping Previous Steps
+### Skipping Previous Steps Method
+
+You should notice that `ufo ship` re-built the docker image and re-registered the task definitions.  The `ufo ship` command is designed to run everything in one simple command, so we do not have to manually call the commands in the previous pages: `ufo build` and `ufo tasks`.
+
+If you would like to skip the first 2 steps, then you can use the [ufo deploy]({% link _docs/ufo-deploy.md %}) instead.  The `ufo deploy` command will:
 
-The `ufo ship` command will automatically calls the steps we called manually in the previous pages: `ufo build` and `ufo tasks`.
+1. register the task definition in `.ufo/output/hi-web.json` unmodified
+2. update the ECS service
 
-If you would like to skip the first 2 steps you can use the `--no-docker` and `--no-tasks` flags:
+Example:
 
 ```sh
-ufo ship hi-web --no-docker --no-tasks
+ufo deploy hi-web
 ```
 
 The output should look something like this:

data/docs/_docs/tutorial-ufo-ships.md

@@ -2,7 +2,7 @@
 title: Deploy Multiple Apps
 ---
 
-You might have noticed in in the tutorial that the generated starter ufo folder contains 3 task definitions a web, worker and clock role.  This is a common pattern.  The web process handles web traffic, the worker process handles background job processing that would be too slow and potentially block web requests, and a clock process is typically used to schedule recurring jobs.
+You might have noticed in in the tutorial that the generated starter ufo folder contains 3 task definitions a `web`, `worker` and `clock` role.  This is a common pattern.  The web process handles web traffic, the worker process handles background job processing that would be too slow and potentially block web requests, and a clock process is typically used to schedule recurring jobs.
 
 These processes use the same codebase and same docker image, but have slightly different run time settings.  The docker run command for a web process could be [puma](http://puma.io/) and the command for a worker process could be [sidekiq](http://sidekiq.org/).  Environment variables are also sometimes different.  The important key is that the same docker image is used for all 3 services but the task definition for each service is slightly different.
 

data/docs/_docs/tutorial-ufo-tasks-build.md

@@ -59,7 +59,9 @@ task_definition "hi-worker" do
 end
 ```
 
-The shared variables are set in the variables folder:
+### Shared Variables
+
+Ufo has a concept of shared variables, covered in [Shared Variables]({% link _docs/variables.md %}). The shared variables are set in the `variables` folder and essentially allow you to use a set of shared variables through your templates:
 
 **ufo/variables/base.rb**:
 
@@ -70,11 +72,11 @@ The shared variables are set in the variables folder:
 @environment = helper.env_file(".env")
 ```
 
-**ufo/variables/prod.rb**:
+**ufo/variables/development.rb**:
 
 ```ruby
 @environment = helper.env_vars(%Q{
-  RAILS_ENV=production
+  RAILS_ENV=development
   SECRET_KEY_BASE=secret
 })
 ```
@@ -97,10 +99,10 @@ You should see output similar to below:
 $ ufo tasks build
 Building Task Definitions...
 Generating Task Definitions:
-  ufo/output/hi-web.json
-  ufo/output/hi-worker.json
-  ufo/output/hi-clock.json
-Task Definitions built in ufo/output.
+  .ufo/output/hi-web.json
+  .ufo/output/hi-worker.json
+  .ufo/output/hi-clock.json
+Task Definitions built in .ufo/output
 $
 ```
 
@@ -112,8 +114,8 @@ Let's take a look at one of the generated files: `ufo/output/hi-web.json`.
   "containerDefinitions": [
     {
       "name": "web",
-      "image": "tongueroo/hi:ufo-2017-09-10T15-13-14-1105f71",
-      "cpu": 128,
+      "image": "tongueroo/hi:ufo-2018-02-13T11-33-15-27aa242",
+      "cpu": 192,
       "memoryReservation": 256,
       "portMappings": [
         {
@@ -121,27 +123,34 @@ Let's take a look at one of the generated files: `ufo/output/hi-web.json`.
           "protocol": "tcp"
         }
       ],
-      "command": [
-        "bin/web"
-      ],
+      "command": null,
       "environment": [
         {
           "name": "RAILS_ENV",
-          "value": "production"
+          "value": "development"
         },
         {
           "name": "SECRET_KEY_BASE",
           "value": "secret"
         }
       ],
+      "logConfiguration": {
+        "logDriver": "awslogs",
+        "options": {
+          "awslogs-group": "ecs/hi-web",
+          "awslogs-region": "us-east-1",
+          "awslogs-stream-prefix": "hi"
+        }
+      },
       "essential": true
     }
   ]
+}
 ```
 
 ### Register the ECS Task Definitions
 
-You have built the ecs task definitions locally on your machine. To register the task definitions in output run this command:
+You have built the ecs task definitions locally on your machine. To register the task definitions in the `output` folder run:
 
 ```sh
 ufo tasks register

data/docs/_docs/ufo-deploy.md

@@ -0,0 +1,30 @@
+---
+title: ufo deploy
+---
+
+It is useful to sometimes deploy only the task definition without re-building it.  Say for example, you are debugging the task definition and just want to directly edit the `.ufo/output/hi-web.json` definition. You can accomplish this with the `ufo deploy` command.  The `ufo deploy` command will deploy the task definition in `.ufo/output` unmodified.  Example:
+
+```
+ufo deploy hi-web
+```
+
+The above command does the following:
+
+1. register the `.ufo/output/hi-web.json` task definition to ECS untouched.
+2. deploys it to ECS by updating the service
+
+The `ufo deploy` command does less than the `ufo ship` command.  Typically, people use [ufo ship]({% link _docs/ufo-ship.md %}) over the `ufo deploy` command do everything in one step:
+
+1. build the Docker image
+2. register the ECS task definition
+3. update the ECS service
+
+The `ufo ships`, `ufo ship`, `ufo deploy` command support the same options. The options are presented here again for convenience:
+
+{% include ufo-ship-options.md %}
+
+
+<a id="prev" class="btn btn-basic" href="{% link _docs/ufo-ships.md %}">Back</a>
+<a id="next" class="btn btn-primary" href="{% link _docs/ufo-scale.md %}">Next Step</a>
+<p class="keyboard-tip">Pro tip: Use the <- and -> arrow keys to move back and forward.</p>
+

data/docs/_docs/ufo-docker-base.md

@@ -8,7 +8,7 @@ Docker is fantastic and has given developers more power and control over the OS
 
 The `ufo docker base` commands allows you to build an Docker image from `Dockerfile.base` to be used as a cache and in the FROM instruction in the main `Dockerfile`.  This base Docker cache image technique speeds up development substantially if you have dependencies like gems in your project.  Let's say you have 20 gems in your project and are actively developing your project. You are experimenting with 3 different gems, adding and removing them while you are investigating the best gem to use.  Without the Docker image cache that already has all of your other gems installed each time you adjust `Gemfile` it would have to wait until all the gems from scratch again installed again.
 
-There are pros and cons of using this approach.  Remember there are 2 hard problems in computer science: 1. Naming and 2. Caching.  The main con about this approach is if you forget to update the base Docker image you will have cached artifacts that will not disappear unless you rebuild the base Docker image.  While some folks are completely against introducing this cache layer, some have found it being a huge win in speeding up their Docker development workflow.  If you are using this technique it is recommended that you set up some automation that at the very least nightly rebuilds the base Docker image.
+There are pros and cons of using this approach.  Remember there are 2 hard problems in computer science: 1. Naming and 2. Caching.  The main con about this approach is if you forget to update the base Docker image you will have cached artifacts that will not disappear unless you rebuild the base Docker image.  While some folks are completely against introducing this cache layer, some have found it being a huge win in speeding up their Docker development workflow.  If you are using this technique it is recommended that you set up some automation that rebuilds the base Docker image at least nightly.
 
 ### Demo
 
@@ -34,9 +34,9 @@ docker build -t tongueroo/hi:base-2017-06-12T14-36-44-2af505e -f Dockerfile.base
 
 It is using the docker `-f Dockerfile.base` option to build the base image.  It names the image with `tongueroo/hi:base-2017-06-12T14-36-44-2af505e`.  The image tag contains useful information: the timestamp when the image was built and the exact git sha of the code.  The image gets push to a registry immediately.
 
-Notice at the very end, the *current* `Dockerfile`'s FROM statement has been updated with the newly built base Docker image automatically.  This saves you from forgettig to copying and pasting it the `Dockerfile` yourself.
+Notice at the very end, the *current* `Dockerfile`'s FROM statement has been updated with the newly built base Docker image automatically.  This saves you from forgetting to copying and pasting it the `Dockerfile` yourself.
 
-<a id="prev" class="btn btn-basic" href="{% link _docs/ufo-docker-build.md %}">Back</a>
+<a id="prev" class="btn btn-basic" href="{% link _docs/ufo-docker-push.md %}">Back</a>
 <a id="next" class="btn btn-primary" href="{% link _docs/ufo-docker-name.md %}">Next Step</a>
 <p class="keyboard-tip">Pro tip: Use the <- and -> arrow keys to move back and forward.</p>
 

data/docs/_docs/ufo-docker-build.md

@@ -2,7 +2,7 @@
 title: ufo docker build
 ---
 
-The `ufo docker build` builds a Docker image using the Dockerfile in the current project folder.  This simply a wrapper command that shells out and calls the `docker` command.  We're use the [tongueroo/hi]((https://github.com/tongueroo/hi) project and it's Dockerfile for demonstration.  Example:
+The `ufo docker build` builds a Docker image using the Dockerfile in the current project folder.  This simply is a wrapper command that shells out and calls the `docker` command.  We're use the [tongueroo/hi]((https://github.com/tongueroo/hi) project and it's Dockerfile for demonstration.  Example:
 
 ```sh
 ufo docker build
@@ -71,9 +71,9 @@ ufo-2017-06-11T22-22-32-a18aa30: digest: sha256:c5385a5084e87643bd943eb120e11032
 Pushed tongueroo/hi:ufo-2017-06-11T22-22-32-a18aa30 docker image. Took 9s.
 ```
 
-Note in order to push the image to a register you will need to login into the registry.  If you are using DockerHub use the `docker login` command.  If you are using AWS ECR then you can use the `aws ecr get-login` command.
+Note in order to push the image to a registry you will need to login into the registry.  If you are using DockerHub use the `docker login` command.  If you are using AWS ECR then you can use the `aws ecr get-login` command.
 
 <a id="prev" class="btn btn-basic" href="{% link _docs/ufo-destroy.md %}">Back</a>
-<a id="next" class="btn btn-primary" href="{% link _docs/ufo-docker-base.md %}">Next Step</a>
+<a id="next" class="btn btn-primary" href="{% link _docs/ufo-docker-push.md %}">Next Step</a>
 <p class="keyboard-tip">Pro tip: Use the <- and -> arrow keys to move back and forward.</p>
 

data/docs/_docs/ufo-docker-push.md

@@ -0,0 +1,43 @@
+---
+title: ufo docker push
+---
+
+The `ufo docker push` command pushes the most recent Docker image built by `ufo docker build` to a registry.  Example:
+
+```sh
+ufo docker build # to build the image
+ufo docker name  # to see the image name
+ufo docker push  # push up the registry
+```
+
+You'll see that `ufo docker push` simply shells out and calls `docker push`:
+
+```
+$ ufo docker push
+=> docker push 123456789.dkr.ecr.us-east-1.amazonaws.com/hi:ufo-2018-02-13T10-51-44-e0cc7be
+The push refers to a repository [123456789.dkr.ecr.us-east-1.amazonaws.com/hi]
+399c739c257d: Layer already exists
+...
+Pushed 123456789.dkr.ecr.us-east-1.amazonaws.com/hi:ufo-2018-02-13T10-51-44-e0cc7be docker image. Took 1s.
+$
+```
+
+You can also specify your own custom image to push as a parameter.
+
+```
+ufo docker push my/image:tag
+```
+
+You could also use the `--push` flag as part of the `ufo docker build` command to achieve the same thing as `ufo docker push`. Some find that `ufo docker push` is more intutitive.
+
+```sh
+ufo docker build --push # same as above
+```
+
+## Docker Authorization
+
+Note in order to push the image to a registry you will need to login into the registry.  If you are using DockerHub use the `docker login` command.  If you are using AWS ECR then, ufo will automatically try to authorize you and configure your `~/.docker/config.json`.  If can also use `aws ecr get-login` command.
+
+<a id="prev" class="btn btn-basic" href="{% link _docs/ufo-docker-build.md %}">Back</a>
+<a id="next" class="btn btn-primary" href="{% link _docs/ufo-docker-base.md %}">Next Step</a>
+<p class="keyboard-tip">Pro tip: Use the <- and -> arrow keys to move back and forward.</p>

data/docs/_docs/ufo-env.md

@@ -2,52 +2,54 @@
 title: UFO_ENV
 ---
 
-Ufo's behavior is controlled by the `UFO_ENV` environment variable.  For example, the `UFO_ENV` variable is used to layer different ufo variable files together to make it easy to specify settings for different environments like production and staging.  This is covered thoroughly in the [Variables]({% link _docs/variables.md %}) section.  `UFO_ENV` defaults to `prod` when not set.
+Ufo's behavior is controlled by the `UFO_ENV` environment variable.  For example, the `UFO_ENV` variable is used to layer different ufo variable files together to make it easy to specify settings for different environments like production and development.  This is covered thoroughly in the [Variables]({% link _docs/variables.md %}) section.  `UFO_ENV` defaults to `development` when not set.
 
 ### Setting UFO_ENV
 
-The `UFO_ENV` can be set easily in several ways:
+The `UFO_ENV` can be set in several ways:
 
 1. At the CLI command invocation - This takes the highest precedence.
 2. Exported as an environment variable to your shell - This takes the second highest precedence.
-3. As a `aws_profile_ufo_env_map` value in your `ufo/settings.yml` file - This takes the lowest precedence.
+3. From the `aws_profiles` setting in your `settings.yml` file - This takes the lowest precedence.
 
 ### At the CLI Command
 
 ```sh
-UFO_ENV=production ufo ship hi-web --cluster prod
+ufo ship hi-web --cluster production
 ```
 
 ### As an environment variable
 
 ```sh
 export UFO_ENV=production
-ufo ship hi-web --cluster prod
+ufo ship hi-web
 ```
 
 Most people will set `UFO_ENV` in their `~/.profile`.
 
 ### In ufo/settings.yml
 
-The most interesting way to set `UFO_ENV` is with the `aws_profile_ufo_env_map` in `ufo/settings.yml`.  Let's say you have a `~/.ufo/settings.yml` with the following:
+The most interesting way to set `UFO_ENV` is with the `aws_profiles` setting in `ufo/settings.yml`.  Let's say you have a `~/.ufo/settings.yml` with the following:
 
 ```yaml
-aws_profile_ufo_env_map:
-  default: development
-  my-prod-profile: production
-  my-stag-profile: staging
+development:
+  aws_profiles:
+    - my-dev-profile
+
+production:
+  aws_profiles:
+    - my-prod-profile
 ```
 
-In this case, when you set `AWS_PROFILE` to switch AWS profiles, ufo picks this up and maps the `AWS_PROFILE` value to the specified `UFO_ENV` using the `aws_profile_ufo_env_map` lookup.  Example:
+In this case, when you set `AWS_PROFILE` to switch AWS profiles, ufo picks this up and maps the `AWS_PROFILE` value to the specified `UFO_ENV` using the `aws_profiles` lookup.  Example:
 
 ```sh
 AWS_PROFILE=my-prod-profile => UFO_ENV=production
-AWS_PROFILE=my-stag-profile => UFO_ENV=staging
-AWS_PROFILE=default => UFO_ENV=development
-AWS_PROFILE=whatever => UFO_ENV=development
+AWS_PROFILE=my-dev-profile => UFO_ENV=development
+AWS_PROFILE=whatever => UFO_ENV=development # since there are no profiles that match
 ```
 
-Notice how `AWS_PROFILE=whatever` results in `UFO_ENV=development`.  This is because the `default: development` map is specially treated. If you set the `default` map, this becomes the default value when the profile map is not specified in the rest of `ufo/settings.yml`.  More info on settings is available at [settings]({% link _docs/settings.md %}).
+Notice how `AWS_PROFILE=whatever` results in `UFO_ENV=development`.  This is because there are not matching aws_profiles in the `settings.yml`.  More info on settings is available at [settings]({% link _docs/settings.md %}).
 
 <a id="prev" class="btn btn-basic" href="{% link _docs/settings.md %}">Back</a>
 <a id="next" class="btn btn-primary" href="{% link _docs/variables.md %}">Next Step</a>