ufo 5.0.0 → 5.0.5

data/docs/_docs/settings/network.md

@@ -2,7 +2,7 @@
 title: Settings Network
 short_title: Network
 categories: settings
-nav_order: 16
+nav_order: 17
 ---
 
 The settings.yml file references a network settings file with the `network_profile` option. This file has configurations that are related to the network.  The source code for the starter template file is at [network/default.yml.tt](https://github.com/tongueroo/ufo/blob/master/lib/template/.ufo/settings/network/default.yml.tt)  Here's an example network settings file.
@@ -22,6 +22,16 @@ elb_subnets: # defaults to same subnets as ecs_subnets when not set
 #   - sg-aaa
 # ecs_security_groups:
 #   - sg-bbb
+
+# Also supports extra security groups specific to each ECS service.
+# So you can target security groups on a per-role basis.
+# ecs_security_groups:
+#   demo-web:
+#   - sg-bbb
+#   - sg-ccc
+#   demo-worker:
+#   - sg-bbb
+#   - sg-ccc
 ```
 
 Option | Description

data/docs/_docs/structure.md

@@ -25,15 +25,16 @@ The table below covers the purpose of each folder and file.
 
 File / Directory  | Description
 ------------- | -------------
-<code>output/</code>  | The folder where the generated task definitions are written to.  The way the task definitions are generated is covered in [ufo tasks build]({% link _docs/tutorial-ufo-tasks-build.md %}).
-<code>params</code>  | This is where you can adjust the params that get send to the aws-sdk api calls for the [ufo task](https://ufoships.com/reference/ufo-task/) command. More info at [Params]({% link _docs/ufo-task-params.md %}).
-<code>settings.yml</code>  | Ufo's general settings file, where you adjust the default [settings]({% link _docs/settings.md %}).
-<code>settings/cfn/default.yml</code>  | Ufo's cfn settings. You can customize the CloudFormation resource properties here.
-<code>settings/network/default.yml</code>  | Ufo's network settings. You can customize the vpc and subnets to used here.
-<code>task_definitions.rb</code>  | This is where you define the task definitions and specify the variables to be used by the ERB templates.
-<code>templates/</code>  | The ERB templates with the task definition json code.  The templates are covered in more detail in [ufo tasks build]({% link _docs/tutorial-ufo-tasks-build.md %}).
-<code>templates/main.json.erb</code>  | This is the main and starter template task definition json file that ufo initially generates.
-<code>variables</code>  | This is where you can define shared variables that are made available to the `template_definitions.rb` and your templates. More info at [Variables]({% link _docs/variables.md %}).
+iam_roles/  | Where ufo managed iam roles associated with the task definition can be defined. For more details see: [IAM Roles]({% link _docs/iam-roles.md %}).
+output/  | The folder where the generated task definitions are written to.  The way the task definitions are generated is covered in [ufo tasks build]({% link _docs/tutorial-ufo-tasks-build.md %}).
+params  | This is where you can adjust the params that get send to the aws-sdk api calls for the [ufo task](https://ufoships.com/reference/ufo-task/) command. More info at [Params]({% link _docs/ufo-task-params.md %}).
+settings.yml  | Ufo's general settings file, where you adjust the default [settings]({% link _docs/settings.md %}).
+settings/cfn/default.yml  | Ufo's cfn settings. You can customize the CloudFormation resource properties here.
+settings/network/default.yml  | Ufo's network settings. You can customize the vpc and subnets to used here.
+task_definitions.rb  | This is where you define the task definitions and specify the variables to be used by the ERB templates.
+templates/  | The ERB templates with the task definition json code.  The templates are covered in more detail in [ufo tasks build]({% link _docs/tutorial-ufo-tasks-build.md %}).
+templates/main.json.erb  | This is the main and starter template task definition json file that ufo initially generates.
+variables  | This is where you can define shared variables that are made available to the `template_definitions.rb` and your templates. More info at [Variables]({% link _docs/variables.md %}).
 
 Now that you know where the ufo configurations are located and what they look like, let’s use ufo!
 

data/docs/_docs/tutorial-ufo-init.md

@@ -65,8 +65,6 @@ base:
   # clean_keep: 30 # cleans up docker images on your docker server.
   # ecr_keep: 30 # cleans up images on ECR and keeps this remaining amount. Defaults to keep all.
   # defaults when an new ECS service is created by ufo ship
-  network_profile: default # .ufo/settings/network/default.yml file
-  cfn_profile: default # .ufo/settings/cfn/default.yml file
 
 development:
   # cluster: dev # uncomment if you want the cluster name be other than the default
@@ -75,14 +73,10 @@ development:
   # When you have AWS_PROFILE set to one of these values, ufo will switch to the desired
   # environment. This prevents you from switching AWS_PROFILE, forgetting to
   # also switch UFO_ENV, and accidentally deploying to production vs development.
-  # aws_profiles:
-  #   - dev_profile1
-  #   - dev_profile2
+  # aws_profile: dev_profile1
 
 production:
   # cluster: prod
-  # aws_profiles:
-  #   - prod_profile
 ```
 
 The `image` value is the name that ufo will use as a base portion of the name to generate a Docker image name, it should not include the tag portion.

data/docs/_docs/ufo-current.md

@@ -1,6 +1,6 @@
 ---
 title: Ufo Current
-nav_order: 23
+nav_order: 26
 ---
 
 ## service

data/docs/_docs/ufo-env-extra.md

@@ -1,6 +1,6 @@
 ---
 title: UFO_ENV_EXTRA
-nav_order: 22
+nav_order: 25
 ---
 
 <div class="video-box"><div class="video-container"><iframe src="https://www.youtube.com/embed/UVQuwQGToYE" frameborder="0" allowfullscreen=""></iframe></div></div>

data/docs/_docs/ufo-env.md

@@ -1,6 +1,6 @@
 ---
 title: UFO_ENV
-nav_order: 21
+nav_order: 24
 ---
 
 Ufo's behavior is controlled by the `UFO_ENV` environment variable.  For example, the `UFO_ENV` variable is used to layer different ufo variable files together to make it easy to specify settings for different environments like production and development.  This is covered thoroughly in the [Variables]({% link _docs/variables.md %}) section.  `UFO_ENV` defaults to `development` when not set.
@@ -27,12 +27,10 @@ The most interesting way to set `UFO_ENV` is with the `aws_profiles` setting in
 
 ```yaml
 development:
-  aws_profiles:
-    - my-dev-profile
+  aws_profile: my-dev-profile
 
 production:
-  aws_profiles:
-    - my-prod-profile
+  aws_profile: my-prod-profile
 ```
 
 In this case, when you set `AWS_PROFILE` to switch AWS profiles, ufo picks this up and maps the `AWS_PROFILE` value to the specified `UFO_ENV` using the `aws_profiles` lookup.  Example:

data/docs/_docs/ufo-logs.md

@@ -1,6 +1,6 @@
 ---
 title: ufo logs command
-nav_order: 20
+nav_order: 23
 ---
 
 The ufo logs command will tail the logs of the ecs service if you are using the awslogs driver.
@@ -46,5 +46,4 @@ The generated .ufo task definition defaults to the awslogs driver. If you need i
 }
 ```
 
-
 {% include prev_next.md %}

data/docs/_docs/ufo-task-params.md

@@ -1,6 +1,6 @@
 ---
 title: Ufo Task Params
-nav_order: 24
+nav_order: 27
 ---
 
 You can run one off task with the [ufo task](https://ufoships.com/reference/ufo-task/) command.

data/docs/_docs/upgrading.md

@@ -1,6 +1,6 @@
 ---
 title: Upgrading
-nav_order: 34
+nav_order: 38
 ---
 
 <ul>

data/docs/_docs/upgrading/upgrade4.5.md

@@ -1,9 +1,9 @@
 ---
 title: Upgrading to Version 4.5
 short_title: Version 4.5
-order: 1
+order: 2
 categories: upgrading
-nav_order: 35
+nav_order: 40
 ---
 
 In ufo version 4.4 and 4.5, the default cloudformation stack names used by ufo were changed.

data/docs/_docs/upgrading/upgrade4.md

@@ -1,9 +1,9 @@
 ---
 title: Upgrading to Version 4.0
 short_title: Version 4.0
-order: 2
+order: 3
 categories: upgrading
-nav_order: 36
+nav_order: 41
 ---
 
 A major change in ufo from version 3 to 4 is that the ECS service is now created by CloudFormation. If you have an existing ECS service deployed by ufo version 3, when you deploy your app again with ufo version 4, there will be a new additional ECS service created. Here is the recommended upgrade path.

data/docs/_docs/upgrading/upgrade5.md

@@ -0,0 +1,19 @@
+---
+title: Upgrading to Version 5
+short_title: Version 5
+order: 1
+categories: upgrading
+nav_order: 39
+---
+
+In ufo v5, the ufo went from underscore key names in the [cfn settings files]({% link _docs/settings/cfn.md %}) to camelized key names. So the auto_camelize behavior is disabled for newly `ufo init` projects. This mean ufo is backwards compatiable. You can enable the v5 behavior with `auto_camelize: false`. If you have not adjusted this setting, then ufo should still work with your current `.ufo` files.
+
+## Upgrading Instructions
+
+If you want to upgrade to the latest ufo v5 default behavior.
+
+1. Adjust your .ufo/settings/cfn files so that the keys are camelized
+2. Add to your .ufo/settings.yml `auto_camelize: false`
+3. Deploy and verify that your ECS app stil works
+
+{% include prev_next.md %}

data/docs/_docs/variables.md

@@ -1,6 +1,6 @@
 ---
 title: Shared Variables
-nav_order: 17
+nav_order: 18
 ---
 
 Often, you end up using the set of common variables across your task definitions for a project.  Ufo supports a shared variables concept to support this.  You specify variables files in the `.ufo/variables` folder and they are made available to your `.ufo/task_definitions.rb` as well as your `.ufo/templates` files.

data/docs/_includes/cfn-customize.md

@@ -5,11 +5,11 @@ The properties in the file `.ufo/settings/cfn/default.yml` map directly to ufo's
 Let's customize the `AWS::ElasticLoadBalancingV2::TargetGroup` resource created by CloudFormation.  We'll adjust the `deregistration_delay.timeout_seconds` to `8`.  Here's the relevant section of the `.ufo/settings/cfn/default.yml`
 
 ```
-target_group:
+TargetGroup:
 ...
-  target_group_attributes:
-  - key: deregistration_delay.timeout_seconds
-    value: 8
+  TargetGroupAttributes:
+  - Key: deregistration_delay.timeout_seconds
+    Value: 8
 ```
 
 The value will be injected to the generated CloudFormation template under the corresponding "TargetGroup Properties".  The generated template looks something like this:
@@ -36,4 +36,18 @@ Here's a list of the resources in the [cfn/stack.yml](https://github.com/tonguer
 * Listener
 * TargetGroup
 
+## Layering Support
+
+The `base.yml` profile files always get evaluated and env-specific ENV.yml file are layered or merged together. This miminizes duplication.  For example, these files are merged:
+
+* .ufo/settings/cfn/base.yml
+* .ufo/settings/cfn/development.yml
+
+The settings in `development.yml` override the settings in `base.yml`. Here's another example:
+
+* .ufo/settings/network/base.yml
+* .ufo/settings/network/development.yml
+
+Note, this feature is available in v5.0.5+.
+
 For the most up to date list check out the [cfn/stack.yml](https://github.com/tongueroo/ufo/blob/master/lib/cfn/stack.yml) source code directly.

data/docs/_includes/footer.html

@@ -6,11 +6,12 @@
                     <div class="footer-col col-md-4">
                         <h3>More Tools</h3>
                         <ul class="list-unstyled tools">
-                            <li><a href="http://rubyonjets.com">Jets</a></li>
-                            <li><a href="http://lono.cloud">Lono</a></li>
-                            <li><a href="http://sonic-screwdriver.cloud">Sonic</a></li>
-                            <li><a href="http://jack-eb.com">Jack</a></li>
-                            <li><a href="https://boltops.com/toolbelt">Toolbelt</a></li>
+                            <li><a href="https://terraspace.cloud">Terraspace</a></li>
+                            <li><a href="https://kubes.guru">Kubes</a></li>
+                            <li><a href="https://rubyonjets.com">Jets</a></li>
+                            <li><a href="https://lono.cloud">Lono</a></li>
+                            <li><a href="https://sonic-screwdriver.cloud">Sonic</a></li>
+                            <li><a href="https://jack-eb.com">Jack</a></li>
                         </ul>
                     </div>
                     <div class="footer-col col-md-4">

data/docs/_reference/ufo-init.md

@@ -107,20 +107,19 @@ If you would like to use a local template that is not on GitHub, then created a
 ## Options
 
 ```
-[--force]                                  # Bypass overwrite are you sure prompt for existing files.
---image=IMAGE                              # Docker image name without the tag. Example: tongueroo/demo-ufo. Configures ufo/settings.yml
-[--app=APP]                                # App name. Preferably one word. Used in the generated ufo/task_definitions.rb.  If not specified then the app name is inferred as the folder name.
-[--launch-type=LAUNCH_TYPE]                # ec2 or fargate.
-                                           # Default: ec2
-[--execution-role-arn=EXECUTION_ROLE_ARN]  # execution role arn used by tasks, required for fargate.
-[--template=TEMPLATE]                      # Custom template to use.
-[--template-mode=TEMPLATE_MODE]            # Template mode: replace or additive.
-[--vpc-id=VPC_ID]                          # Vpc id. For settings/network/default.yml.
-[--ecs-subnets=one two three]              # Subnets for ECS tasks, defaults to --elb-subnets set to. For settings/network/default.yml
-[--elb-subnets=one two three]              # Subnets for ELB. For settings/network/default.yml
-[--verbose], [--no-verbose]                
-[--mute], [--no-mute]                      
-[--noop], [--no-noop]                      
-[--cluster=CLUSTER]                        # Cluster.  Overrides .ufo/settings.yml.
+[--force]                        # Bypass overwrite are you sure prompt for existing files.
+--image=IMAGE                    # Docker image name without the tag. Example: tongueroo/demo-ufo. Configures ufo/settings.yml
+[--app=APP]                      # App name. Preferably one word. Used in the generated ufo/task_definitions.rb.  If not specified then the app name is inferred as the folder name.
+[--launch-type=LAUNCH_TYPE]      # ec2 or fargate.
+                                 # Default: ec2
+[--template=TEMPLATE]            # Custom template to use.
+[--template-mode=TEMPLATE_MODE]  # Template mode: replace or additive.
+[--vpc-id=VPC_ID]                # Vpc id. For settings/network/default.yml.
+[--ecs-subnets=one two three]    # Subnets for ECS tasks, defaults to --elb-subnets set to. For settings/network/default.yml
+[--elb-subnets=one two three]    # Subnets for ELB. For settings/network/default.yml
+[--verbose], [--no-verbose]      
+[--mute], [--no-mute]            
+[--noop], [--no-noop]            
+[--cluster=CLUSTER]              # Cluster.  Overrides .ufo/settings.yml.
 ```
 

data/docs/articles.md

@@ -1,6 +1,6 @@
 ---
 title: Articles
-nav_order: 46
+nav_order: 51
 ---
 
 * [How to Create Unlimited Extra Environments

data/lib/template/.secrets

@@ -1,3 +1,5 @@
-# fine to have comment in this file
-NAME1=SSM:parameter_name
-NAME2=SECRETSMANAGER:secret_name-AbCdEf
+# Example starter secrets file. Be sure that the SSM parameters or Secrets exist.
+# Docs: https://ufoships.com/docs/secrets/
+#
+# NAME1=SSM:parameter_name
+# NAME2=SECRETSMANAGER:secret_name-AbCdEf

data/lib/template/.ufo/iam_roles/execution_role.rb

@@ -0,0 +1,7 @@
+# Example starter execution role. Add the iam role permissions that the host needs here:
+#
+# More docs: https://ufoships.com/docs/iam-roles/
+#
+managed_iam_policy("AmazonSSMReadOnlyAccess")
+managed_iam_policy("SecretsManagerReadWrite")
+managed_iam_policy("service-role/AmazonECSTaskExecutionRolePolicy")

data/lib/template/.ufo/iam_roles/task_role.rb

@@ -0,0 +1,21 @@
+# Example starter task role. Add the iam role permissions that the container needs here:
+#
+# More docs: https://ufoships.com/docs/iam-roles/
+#
+# Examples:
+#
+# iam_policy("AmazonS3ReadOnlyAccess",
+#   Action: [
+#     "s3:Get*",
+#     "s3:List*"
+#   ],
+#   Effect: "Allow",
+#   Resource: "*"
+# )
+# iam_policy("CloudwatchWrite",
+#   Action: [
+#     "cloudwatch:PutMetricData",
+#   ],
+#   Effect: "Allow",
+#   Resource: "*"
+# )

data/lib/template/.ufo/templates/fargate.json.erb

@@ -2,7 +2,6 @@
   "family": "<%= @family %>",
   "requiresCompatibilities": ["FARGATE"],
   "networkMode": "awsvpc",
-  "executionRoleArn": "<%= @execution_role_arn || raise("@execution_role_arn needs to be set") %>",
   "cpu": "<%= @cpu %>",
   "memory": "<%= @memory %>",
   "containerDefinitions": [

data/lib/ufo/dsl/helper.rb

@@ -46,11 +46,11 @@ module Ufo
       end
 
       def secrets(text)
-        Vars.new(text: text, secrets: true).secrets
+        Vars.new(text: text).secrets
       end
 
       def secrets_file(path)
-        Vars.new(file: path, secrets: true).secrets
+        Vars.new(file: path).secrets
       end
 
       def current_region

data/lib/ufo/dsl/helper/vars.rb

@@ -8,7 +8,6 @@ class Ufo::DSL::Helper
       # use either file or text. text takes higher precedence
       @file = options[:file]
       @text = options[:text]
-      @secrets = options[:secrets] # true or false
     end
 
     def content

data/lib/ufo/ecr/auth.rb

@@ -1,3 +1,5 @@
+require 'open3'
+
 =begin
 Normally, you must authorized to AWS ECR to push to their registry with:
 
@@ -27,19 +29,15 @@ module Ufo
       return unless ecr_image?
 
       auth_token = fetch_auth_token
-      if File.exist?(docker_config)
-        data = JSON.load(IO.read(docker_config))
-        data["auths"][@repo_domain] = {auth: auth_token}
-      else
-        data = {"auths" => {@repo_domain => {auth: auth_token}}}
+      username, password = Base64.decode64(auth_token).split(':')
+
+      command = "docker login -u #{username} --password-stdin #{@repo_domain}"
+      puts "=> #{command}".color(:green)
+      *, status = Open3.capture3(command, stdin_data: password)
+      unless status.success?
+        puts "ERROR: The docker failed to login.".color(:red)
+        exit 1
       end
-
-      # Handle legacy docker clients that still have old format with https://
-      legacy_entry = "https://#{@repo_domain}"
-      data["auths"][legacy_entry] = {auth: auth_token}
-
-      ensure_dotdocker_exists
-      IO.write(docker_config, JSON.pretty_generate(data))
     end
 
     def ecr_image?
@@ -50,14 +48,5 @@ module Ufo
       ecr.get_authorization_token.authorization_data.first.authorization_token
     end
 
-    def docker_config
-      "#{ENV['HOME']}/.docker/config.json"
-    end
-
-    def ensure_dotdocker_exists
-      dirname = File.dirname(docker_config)
-      FileUtils.mkdir_p(dirname) unless File.exist?(dirname)
-    end
-
   end
 end

data/lib/ufo/init.rb

@@ -9,7 +9,6 @@ module Ufo
         [:image, required: true, desc: "Docker image name without the tag. Example: tongueroo/demo-ufo. Configures ufo/settings.yml"],
         [:app, desc: "App name. Preferably one word. Used in the generated ufo/task_definitions.rb.  If not specified then the app name is inferred as the folder name."],
         [:launch_type, default: "ec2", desc: "ec2 or fargate."],
-        [:execution_role_arn, desc: "execution role arn used by tasks, required for fargate."],
         [:template, desc: "Custom template to use."],
         [:template_mode, desc: "Template mode: replace or additive."],
         [:vpc_id, desc: "Vpc id. For settings/network/default.yml."],
@@ -56,7 +55,6 @@ module Ufo
       # map variables
       @app = options[:app] || inferred_app
       @image = options[:image]
-      @execution_role_arn_input = get_execution_role_arn_input
       # copy the files
       puts "Setting up ufo project..."
       exclude_pattern = File.exist?("#{Ufo.root}/Dockerfile") ?

data/lib/ufo/sequence.rb

@@ -17,22 +17,6 @@ module Ufo
       File.basename(Dir.pwd)
     end
 
-    def get_execution_role_arn_input
-      return @execution_role_arn if @execution_role_arn
-
-      if @options[:execution_role_arn]
-        @execution_role_arn = @options[:execution_role_arn]
-        return @execution_role_arn
-      end
-
-      return unless @options[:launch_type] == "fargate"
-      # execution role arn required for fargate
-      puts "For fargate ECS tasks an ECS Task Execution IAM Role is required. "
-      puts "More details here: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_execution_IAM_role.html"
-      print "Please provide a execution role arn role for the ecs task: "
-      @execution_role_arn = $stdin.gets.strip
-    end
-
     def override_source_paths(*paths)
       # Using string with instance_eval because block doesnt have access to
       # path at runtime.