ufo 5.0.0 → 5.0.5

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ce5c8180b261636a61805a4abc5cbd3d556784f77dbf20ca0f2384d8ab50ae32
-  data.tar.gz: 9d6e1955bd7ca4b35b347c61986af5c92c916410a1d9b43b0d11a76e1dbd4fc3
+  metadata.gz: ac9da556f0438367ebd761b2f0c90e04b175bee2b7c6d9deb7fd06d4d05fa755
+  data.tar.gz: a5ae0d961c966ab386981a81e41e184e9a4821e59c8812ec68052a053372123d
 SHA512:
-  metadata.gz: 60ec0e82534f94e8daffbb9587f22753e4df92a77545ba4f220e0f9f3f1568a7dc0722f32e278f1853160ab477e80b0cfef8dbde6330c0b0af46a3e4695c8bc6
-  data.tar.gz: 4bb7540d47f271ea211b3ade315b6a69fe9d1ce23b2cf0cef82cf2ddb1905d849a6064a08aab07279e623073f985542f8396533aedde86fc0b78a39e99ca8bb0
+  metadata.gz: 6a679e2f13efe4d64f34b66ba7d2f4a859abe560451fdb722f0e41919fecbddb4ed8d3ef4047fbd1745b7a6dd381af6c075fb95f7355f46a751866bfa3e79f07
+  data.tar.gz: 4b550246bf57f094add49eaf7171268c8c20f6f6a13d40f7b72f3834c0aa5f023cbff0802f313b8e7c3821ec51da60638d726a92ab6f8ae241cc7f11dfcb22b1

data/CHANGELOG.md

@@ -3,6 +3,24 @@
 All notable changes to this project will be documented in this file.
 This project *tries* to adhere to [Semantic Versioning](http://semver.org/), even before v1.0.
 
+## [5.0.5] - 2021-01-23
+- allow base profile without a default profile
+
+## [5.0.4] - 2021-01-23
+- [#119](https://github.com/tongueroo/ufo/pull/119) layer base profiles with env-specific or default profile
+
+## [5.0.3] - 2020-12-10
+- [#118](https://github.com/tongueroo/ufo/pull/118) update aws-mfa-secure with require singleton fix
+
+## [5.0.2]
+- #111 Add support of credsStore
+- #112 Add support for bridge network mode
+- #113 Allow custom container name when you try to attach an existing ELB to a service
+
+## [5.0.1]
+- #109 fix fargate
+- #110 adjust and document `managed_security_groups` setting
+
 ## [5.0.0]
 - #104 adjust logs default format to detailed
 - #105 major rework: build cfn template with Ruby instead of ERB for new features

data/docs/_docs/conventions.md

@@ -1,6 +1,6 @@
 ---
 title: Conventions
-nav_order: 19
+nav_order: 22
 ---
 
 Ufo uses a set of naming conventions.  This helps enforce some best practices and also allows the ufo commands to be concise.  You can override or bypass the conventions easily.

data/docs/_docs/extras/codebuild-iam-role.md

@@ -1,6 +1,6 @@
 ---
 title: CodeBuild IAM Role
-nav_order: 32
+nav_order: 35
 ---
 
 Note, the `/tmp/ecs-deploy-policy.json` policy is available at [Minimal Deploy IAM]({% link _docs/extras/minimal-deploy-iam.md %}).

data/docs/_docs/extras/dockerfile-erb.md

@@ -1,6 +1,6 @@
 ---
 title: Dynamic Dockerfile.erb
-nav_order: 33
+nav_order: 36
 ---
 
 Sometimes you may need a little more dynamic control of your Dockerfile. For these cases, ufo supports dynamically creating a Dockerfile from a Dockerfile.erb.  If Dockerfile.erb exists, ufo uses it to generate a Dockerfile as a part of the build process.  These means that you should update the source Dockerfile.erb instead, as the Dockerfile will be overwritten.  If Dockerfile.erb does not exist, then ufo will use the Dockerfile instead.

data/docs/_docs/extras/ecs-network-mode.md

@@ -1,6 +1,6 @@
 ---
 title: ECS Network Mode
-nav_order: 27
+nav_order: 30
 ---
 
 ## Pros and Cons: bridge network mode

data/docs/_docs/extras/load-balancer.md

@@ -1,6 +1,6 @@
 ---
 title: Load Balancer Support
-nav_order: 25
+nav_order: 28
 ---
 
 Ufo can automatically create a load balancer and associate it with an ECS service.  The options:

data/docs/_docs/extras/minimal-deploy-iam.md

@@ -1,6 +1,6 @@
 ---
 title: Minimal Deploy IAM Policy
-nav_order: 31
+nav_order: 34
 ---
 
 The IAM user you use to run the `ufo ship` command needs a minimal set of IAM policies in order to deploy to ECS. Here is a table of the baseline services needed:

data/docs/_docs/extras/notification-arns.md

@@ -1,7 +1,7 @@
 ---
 title: Notification ARNs
 categories: extras
-nav_order: 99
+nav_order: 37
 ---
 
 You can specific notification arns for CloudFormation stack related events with [configs/settings.yml]({% link _docs/settings.md %}). This may be useful for compliance purposes.

data/docs/_docs/extras/redirection-support.md

@@ -1,6 +1,6 @@
 ---
 title: Redirection Support
-nav_order: 30
+nav_order: 33
 ---
 
 ## Application Load Balancers
@@ -8,15 +8,15 @@ nav_order: 30
 If you are using an Application Load Balancer you can configure redirection by editing the default actions of the regular listener that is set up by ufo. This assumes you have set up [SSL Support]({% link _docs/extras/ssl-support.md %}).  Here's an example that redirects http to https with a 302 http status code:
 
 ```
-listener:
-  port: 80
+Listener:
+  Port: 80
   # ...
-  default_actions:
-   - type: redirect
-     redirect_config:
-       protocol: HTTPS
-       status_code: HTTP_302 # HTTP_301 and HTTP_302 are valid
-       port: 443
+  DefaultActions:
+   - Type: redirect
+     RedirectConfig:
+       Protocol: HTTPS
+       StatusCode: HTTP_302 # HTTP_301 and HTTP_302 are valid
+       Port: 443
 ```
 
 

data/docs/_docs/extras/route53-support.md

@@ -1,14 +1,14 @@
 ---
 title: Route53 Support
-nav_order: 29
+nav_order: 32
 ---
 
 Ufo can create a "pretty" route53 record and set it's value to the created ELB DNS name. This is done by configuring the `.ufo/settings/cfn/default.yml` file. Example:
 
 ```yaml
-dns:
-  name: "{stack_name}.mydomain.com."
-  hosted_zone_name: mydomain.com. # dont forget the trailing period
+Dns:
+  Name: "{stack_name}.mydomain.com."
+  HostedZoneName: mydomain.com. # dont forget the trailing period
 ```
 
 The `{stack_name}` variable gets substituted with the CloudFormation stack name launched by ufo. So for example:

data/docs/_docs/extras/security-groups.md

@@ -1,6 +1,6 @@
 ---
 title: Security Groups
-nav_order: 26
+nav_order: 29
 ---
 
 Ufo creates and manages two security groups. One for the ELB and one for the ECS tasks.

data/docs/_docs/extras/ssl-support.md

@@ -1,15 +1,15 @@
 ---
 title: SSL Support
-nav_order: 28
+nav_order: 31
 ---
 
 You can configure SSL support by uncomment the `listener_ssl` option in `.ufo/settings/cfn/default.yml`.  Here's an example:
 
 ```
-listener_ssl:
-  port: 443
-  certificates:
-  - certificate_arn: arn:aws:acm:us-east-1:111111111111:certificate/11111111-2222-3333-4444-555555555555
+ListenerSsl:
+  Port: 443
+  Certificates:
+  - CertificateArn: arn:aws:acm:us-east-1:111111111111:certificate/11111111-2222-3333-4444-555555555555
 ```
 
 For the certificate arn, you will need to create a certificate with AWS ACM. To do so, you can follow these instructions: [Request a Public Certificate

data/docs/_docs/faq.md

@@ -1,6 +1,6 @@
 ---
 title: FAQ
-nav_order: 45
+nav_order: 50
 ---
 
 **Q: Is AWS ECS Fargate supported?**

data/docs/_docs/helpers.md

@@ -1,6 +1,6 @@
 ---
 title: Helpers
-nav_order: 18
+nav_order: 19
 ---
 
 The `task_definitions.rb` file has access to helper methods. These helper methods provide useful contextual information about the project.

data/docs/_docs/iam-roles.md

@@ -1,5 +1,6 @@
 ---
 title: Task Definition IAM Roles
+nav_order: 21
 ---
 
 ## What are ECS IAM Roles?
@@ -45,9 +46,7 @@ You then use a DSL to create the IAM roles. Here are examples:
 .ufo/iam_roles/execution_role.rb
 
 ```ruby
-managed_iam_policy("AmazonEC2ContainerRegistryReadOnly")
 managed_iam_policy("AmazonSSMReadOnlyAccess")
-managed_iam_policy("CloudWatchLogsFullAccess")
 managed_iam_policy("SecretsManagerReadWrite")
 managed_iam_policy("service-role/AmazonECSTaskExecutionRolePolicy")
 ```
@@ -109,3 +108,5 @@ You can also assign the task definition `executionRoleArn` with pre-created IAM
   ]
 }
 ```
+
+{% include prev_next.md %}

data/docs/_docs/install.md

@@ -17,16 +17,6 @@ Or you can add ufo to your Gemfile in your project if you are working with a rub
 gem "ufo"
 {% endhighlight %}
 
-## Install with Bolts Toolbelt
-
-If you want to quickly install ufo without having to worry about ufo's dependencies you can install the Bolts Toolbelt which has ufo included.
-
-```sh
-brew cask install boltopslabs/software/bolts
-```
-
-For more information about the Bolts Toolbelt or to get an installer for another operating system visit: [https://boltops.com/toolbelt](https://boltops.com/toolbelt)
-
 ## Dependencies
 
 * Docker: You will need a working version of [Docker](https://docs.docker.com/engine/installation/) installed as ufo shells out and calls the `docker` command.

data/docs/_docs/more/auto-completion.md

@@ -1,6 +1,6 @@
 ---
 title: Auto Completion
-nav_order: 44
+nav_order: 49
 ---
 
 Ufo supports bash auto-completion.  To set it up add the following to your `~/.profile` or `.bashrc`:

data/docs/_docs/more/automated-cleanup.md

@@ -1,6 +1,6 @@
 ---
 title: Automated Clean Up
-nav_order: 43
+nav_order: 48
 ---
 
 Ufo can be configured to automatically clean old images from the ECR registry after the deploy completes by configuring your [settings.yml]({% link _docs/settings.md %}) file like so:

data/docs/_docs/more/customize-cloudformation.md

@@ -1,6 +1,6 @@
 ---
 title: Customize CloudFormation
-nav_order: 38
+nav_order: 43
 ---
 
 Under the hood, ufo creates most of the required resources with a CloudFormation stack.  This includes the ELB, Target Group, Listener, Security Groups, ECS Service, and Route 53 records.  You might need to customize these resources.  Here are the ways to customize the resources that ufo creates.

data/docs/_docs/more/migrations.md

@@ -1,6 +1,6 @@
 ---
 title: Database Migrations
-nav_order: 42
+nav_order: 47
 ---
 
 A common task is to run database migrations with newer code before deploying the code. This is easily achieved with the `ufo task` command. Here's an example:

data/docs/_docs/more/run-in-pieces.md

@@ -1,6 +1,6 @@
 ---
 title: Run in Pieces
-nav_order: 40
+nav_order: 45
 ---
 
 The `ufo ship` command goes through a few stages:

data/docs/_docs/more/single-task.md

@@ -1,6 +1,6 @@
 ---
 title: Run Single Task
-nav_order: 41
+nav_order: 46
 ---
 
 Sometimes you do not want to run a long running `service` but a one time task. Running Rails migrations are an example of a one off task.  Here is an example of how you would run a one time task.

data/docs/_docs/more/stuck-cloudformation.md

@@ -1,6 +1,6 @@
 ---
 title: Stuck CloudFormation
-nav_order: 39
+nav_order: 44
 ---
 
 The CloudFormation stack update or creation can get stuck in a `*_IN_PROGRESS` state for a very long time, like more than an hour.  This happens when you deploy an ECS service that fails to stabilize. Usually, this is an error with the Docker container failing to start up successfully.

data/docs/_docs/more/why-cloudformation.md

@@ -1,6 +1,6 @@
 ---
 title: Why CloudFormation
-nav_order: 37
+nav_order: 42
 ---
 
 Version 3 of ufo was a simpler implementation and did not make use of CloudFormation to create the ECS service. In version 4, ufo uses CloudFormation to create the ECS Service.  This is because ufo became more powerful. Notably, support for Load Balancers was added. With this power, also came added complexity. So the complexity was push onto CloudFormation.  Hence, ECS service is implemented as CloudFormation resource in version 4.

data/docs/_docs/next-steps.md

@@ -1,6 +1,6 @@
 ---
 title: Next Steps
-nav_order: 47
+nav_order: 52
 ---
 
 This concludes the tutorial guide for ufo. Hopefully you are now more comfortable with ufo's basic usage, concepts, and have a feel for the workflow.

data/docs/_docs/secrets.md

@@ -1,10 +1,11 @@
 ---
 title: Secrets
+nav_order: 20
 ---
 
 ## What are Secrets?
 
-[ECS supports injecting secrets or sensitive data](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/specifying-sensitive-data.html) into the the environment as variables.  ECS handles the decryption the secrets go straight from AWS to the ECS task environment. It never passes through the machine calling `ufo ship` IE: your laptop, a deploy server, or CodeBuild, etc.
+[ECS supports injecting secrets or sensitive data](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/specifying-sensitive-data.html) into the the environment as variables.  ECS decrypts the secrets straight from AWS to the ECS task environment. It never passes through the machine calling `ufo ship` IE: your laptop, a deploy server, or CodeBuild, etc.
 
 ECS supports 2 storage backends for secrets:
 
@@ -55,7 +56,7 @@ Ufo supports both forms of secrets. You create a `.secrets` file and can referen
 
 The `.secrets` file is like an env file that will understand a secrets-smart format.  Example:
 
-    NAME1=SSM:/my/parameter_name
+    NAME1=SSM:my/parameter_name
     NAME2=SECRETSMANAGER:/my/secret_name-AbCdEf
 
 The `SSM:` and `SECRETSMANAGER:` prefix will be expanded to the full ARN. You can also just specify the full ARN.
@@ -71,17 +72,29 @@ In turn, this generates:
     "secrets": [
       {
         "name": "NAME1",
-        "valueFrom": "arn:aws:ssm:us-west-2:536766270177:parameter/demo/development/foo"
+        "valueFrom": "arn:aws:ssm:us-west-2:111111111111:parameter/demo/development/foo"
       },
       {
         "name": "NAME2",
-        "valueFrom": "arn:aws:secretsmanager:us-west-2:536766270177:secret:/demo/development/my-secret-test-qRoJel"
+        "valueFrom": "arn:aws:secretsmanager:us-west-2:111111111111:secret:/demo/development/my-secret-test-qRoJel"
       }
     ]
   }]
 }
 ```
 
+## SSM Parameter Names with Leading Slash
+
+If your SSM parameter has a leading slash then do **not** include when using it in the .secrets file. Example:
+
+    aws ssm get-parameter --name /demo/development/foo
+
+So use:
+
+    FOO=SSM:demo/development/foo
+
+The extra slash seems to confuse ECS. For secretsmanager names, you do include the leading slash.
+
 ## Substitution
 
 Ufo also does a simple substition on the value. For example, the `:UFO_ENV` is replaced with the actual value of `UFO_ENV=development`. Example:
@@ -109,4 +122,14 @@ managed_iam_policy("SecretsManagerReadWrite")
 
 More info [ECS IAM Roles]({% link _docs/iam-roles.md %})
 
+## Debugging Tip
+
+Be sure that the secrets exist. If they do not you will see an error like this in the ecs-agent.log:
+
+/var/log/ecs/ecs-agent.log
+
+    level=info time=2020-06-26T00:59:46Z msg="Managed task [arn:aws:ecs:us-west-2:111111111111:task/development/91828be6a02b48f982cd9122db5e39b2]: error transitioning resource [ssmsecret] to [CREATED]: fetching secret data from SSM Parameter Store in us-west-2: invalid parameters: /my-parameter-name" module=task_manager.go
+
+Sometimes there is even no error message in the ecs-agent.log. As a debugging step, try removing all secrets and seeing if that the container will start up.
+
 {% include prev_next.md %}

data/docs/_docs/settings.md

@@ -16,8 +16,8 @@ base:
   image: tongueroo/demo-ufo
   # clean_keep: 30 # cleans up docker images on your docker server.
   # ecr_keep: 30 # cleans up images on ECR and keeps this remaining amount. Defaults to keep all.
-  network_profile: default # .ufo/settings/network/default.yml file
-  cfn_profile: default # .ufo/settings/cfn/default.yml file
+  # network_profile: default # .ufo/settings/network/default.yml file
+  # cfn_profile: default # .ufo/settings/cfn/default.yml file
 
 development:
   # cluster: dev # uncomment if you want the cluster name be other than the default
@@ -35,13 +35,14 @@ The table below covers each setting:
 
 Setting  | Description
 ------------- | -------------
-`aws_profile`  | If you have the `AWS_PROFILE` environment variable set, this will ensure that you are deploying the right `UFO_ENV` to the right AWS environment. It is explained below.
-`cfn_profile` | The name of the cfn profile settings file to use. Maps to .ufo/settings/cfn/NAME.yml file
-`clean_keep`  | Docker images generated from ufo are cleaned up automatically for you at the end of `ufo ship`. This controls how many docker images to keep around. The default is 3.
-`cluster`  | By convention, the ECS cluster that ufo deploys to matches the `UFO_ENV`. If `UFO=development`, then `ufo ship` deploys to the `development` ECS cluster. This is option overrides this convention.
-`ecr_keep`  | If you are using AWS ECR, then the ECR images can also be automatically cleaned up at the end of `ufo ship`. By default this is set to `nil` and all AWS ECR are kept.
-`image`  | The `image` value is the name that ufo will use for the Docker image name to be built.  Only provide the basename part of the image name without the tag because ufo automatically generates the tag for you. For example, `tongueroo/demo-ufo` is correct and `tongueroo/demo-ufo:my-tag` is incorrect.
-`network_profile` | The name of the network profile settings file to use. Maps to .ufo/settings/network/NAME.yml file
+aws_profile  | If you have the `AWS_PROFILE` environment variable set, this will ensure that you are deploying the right `UFO_ENV` to the right AWS environment. It is explained below.
+cfn_profile | The name of the cfn profile settings file to use. Maps to .ufo/settings/cfn/NAME.yml file. Will match an `UFO_ENV` file if it exists. IE: .ufo/settings/cfn/development.yml. Otherwise it defaults to .ufo/settings/cfn/default.yml.
+clean_keep  | Docker images generated from ufo are cleaned up automatically for you at the end of `ufo ship`. This controls how many docker images to keep around. The default is 3.
+cluster  | By convention, the ECS cluster that ufo deploys to matches the `UFO_ENV`. If `UFO=development`, then `ufo ship` deploys to the `development` ECS cluster. This is option overrides this convention.
+ecr_keep  | If you are using AWS ECR, then the ECR images can also be automatically cleaned up at the end of `ufo ship`. By default this is set to `nil` and all AWS ECR are kept.
+image  | The `image` value is the name that ufo will use for the Docker image name to be built.  Only provide the basename part of the image name without the tag because ufo automatically generates the tag for you. For example, `tongueroo/demo-ufo` is correct and `tongueroo/demo-ufo:my-tag` is incorrect.
+managed\_security\_groups | Create managed security groups for application ELBs. Defaults to true. If you disable it with false then no managed security groups will be created by UFO.
+network_profile | The name of the network profile settings file to use. Maps to .ufo/settings/network/NAME.yml file. Will match an `UFO_ENV` file if it exists. IE: .ufo/settings/network/development.yml. Otherwise it defaults to .ufo/settings/network/default.yml.
 
 ## AWS_PROFILE support
 

data/docs/_docs/settings/manage-security-groups.md

@@ -0,0 +1,24 @@
+---
+title: Managed Security Groups
+short_title: Security Groups
+categories: settings
+nav_order: 16
+---
+
+Ufo creates and manages two security groups. One for the ELB and one for the ECS tasks. Details here: [UFO Security Groups]({% link _docs/extras/security-groups.md %}).
+
+You can disable the creation of managed security groups with: `managed_security_groups: false`. Example:
+
+```yaml
+base:
+  image: tongueroo/demo-ufo
+  managed_security_groups: false
+```
+
+## Why?
+
+Security Groups managed by UFO are transient. If you delete the UFO app and recreate it entirely. Any manual changes to the security groups will be lost.
+
+You can precreate security groups and add them generated UFO CloudFormation template, see [Settings Network]({% link _docs/settings/network.md %}). So then you won't lose any manual changes. If you're taking this approach, it's nice to have UFO not create any managed security groups at all. This removes security group clutter.
+
+{% include prev_next.md %}