ufo 4.5.8 → 4.6.1

data/docs/_docs/tutorial.md

@@ -11,4 +11,4 @@ In the next sections, we'll walk through using ufo in detail. We will ufo-ify a
 
 Let's start!
 
-{% include prev_next.md %}
+{% include prev_next.md %}

data/docs/_docs/ufo-current.md

@@ -1,6 +1,6 @@
 ---
 title: Ufo Current
-nav_order: 22
+nav_order: 23
 ---
 
 ## service
@@ -46,4 +46,4 @@ This can be shorten with with current also.
     ufo current --services demo-web demo-worker demo-clock
     ufo ships
 
-{% include prev_next.md %}
+{% include prev_next.md %}

data/docs/_docs/ufo-env-extra.md

@@ -1,6 +1,6 @@
 ---
 title: UFO_ENV_EXTRA
-nav_order: 21
+nav_order: 22
 ---
 
 <div class="video-box"><div class="video-container"><iframe src="https://www.youtube.com/embed/UVQuwQGToYE" frameborder="0" allowfullscreen=""></iframe></div></div>
@@ -19,4 +19,4 @@ The precedence:
 1. UFO_ENV_EXTRA - takes highest precedence
 2. `.ufo/current` env-extra setting - takes lower precedence
 
-{% include prev_next.md %}
+{% include prev_next.md %}

data/docs/_docs/ufo-env.md

@@ -1,6 +1,6 @@
 ---
 title: UFO_ENV
-nav_order: 20
+nav_order: 21
 ---
 
 Ufo's behavior is controlled by the `UFO_ENV` environment variable.  For example, the `UFO_ENV` variable is used to layer different ufo variable files together to make it easy to specify settings for different environments like production and development.  This is covered thoroughly in the [Variables]({% link _docs/variables.md %}) section.  `UFO_ENV` defaults to `development` when not set.
@@ -45,4 +45,4 @@ AWS_PROFILE=whatever => UFO_ENV=development # since there are no profiles that m
 
 Notice how `AWS_PROFILE=whatever` results in `UFO_ENV=development`.  This is because there are no matching aws_profiles in the `settings.yml`.  More info on settings is available at [settings]({% link _docs/settings.md %}).
 
-{% include prev_next.md %}
+{% include prev_next.md %}

data/docs/_docs/ufo-logs.md

@@ -0,0 +1,50 @@
+---
+title: ufo logs command
+nav_order: 20
+---
+
+The ufo logs command will tail the logs of the ecs service if you are using the awslogs driver.
+
+## Examples
+
+    $ ufo logs demo-web
+    2020-01-16 23:58:16 UTC 10.20.120.135 - - [16/Jan/2020:23:58:16 +0000] "GET / HTTP/1.1" 200 3 0.0003
+    2020-01-16 23:58:16 UTC 10.20.120.135 - - [16/Jan/2020:23:58:16 UTC] "GET / HTTP/1.1" 200 3
+    2020-01-16 23:58:16 UTC - -> /
+
+## Current Set
+
+If you have a current service name set.
+
+    $ ufo current --service demo-web
+    $ ufo logs # follow by default
+    2020-01-16 23:58:16 UTC 10.20.120.135 - - [16/Jan/2020:23:58:16 +0000] "GET / HTTP/1.1" 200 3 0.0003
+    2020-01-16 23:58:16 UTC 10.20.120.135 - - [16/Jan/2020:23:58:16 UTC] "GET / HTTP/1.1" 200 3
+    2020-01-16 23:58:16 UTC - -> /
+
+## Options
+
+By default the logs follow and use the simple format without the log stream. Here's how adjust those options:
+
+    ufo logs --no-follow
+    ufo logs --format detailed # to show stream too
+
+More info: [ufo logs reference]({% link _reference/ufo-logs.md %})
+
+## awslog driver
+
+The generated .ufo task definition defaults to the awslogs driver. If you need it, it looks like this:
+
+```json
+"logConfiguration": {
+  "logDriver": "awslogs",
+  "options": {
+    "awslogs-group": "<%= @awslogs_group %>",
+    "awslogs-region": "<%= @awslogs_region || 'us-east-1' %>",
+    "awslogs-stream-prefix": "<%= @awslogs_stream_prefix %>"
+  }
+}
+```
+
+
+{% include prev_next.md %}

data/docs/_docs/ufo-task-params.md

@@ -1,6 +1,6 @@
 ---
 title: Ufo Task Params
-nav_order: 23
+nav_order: 24
 ---
 
 You can run one off task with the [ufo task](https://ufoships.com/reference/ufo-task/) command.
@@ -40,4 +40,4 @@ ERB and [shared variables]({% link _docs/variables.md %}) are available in the p
 
 Note, the params.yml file does not have access to the `task_definition_name` helper method. That is only available in the `task_definitions.rb` template_definition code blocks.
 
-{% include prev_next.md %}
+{% include prev_next.md %}

data/docs/_docs/upgrading.md

@@ -1,6 +1,6 @@
 ---
 title: Upgrading
-nav_order: 33
+nav_order: 34
 ---
 
 <ul>
@@ -10,4 +10,4 @@ nav_order: 33
 {% endfor %}
 </ul>
 
-{% include prev_next.md %}
+{% include prev_next.md %}

data/docs/_docs/upgrading/upgrade4.5.md

@@ -3,7 +3,7 @@ title: Upgrading to Version 4.5
 short_title: Version 4.5
 order: 1
 categories: upgrading
-nav_order: 34
+nav_order: 35
 ---
 
 In ufo version 4.4 and 4.5, the default cloudformation stack names used by ufo were changed.
@@ -49,4 +49,4 @@ Version | Setting
 
 CloudFormation names the resources it creates with the beginning portion of the stack name. When the stack name prepends the environment then resources like ELBs a little bit harder to identify since they might be named something like this `product-Elb-K0LFFQ9LK50W`. It makes it harder to distinguish ELBs from different apps created by ufo.
 
-{% include prev_next.md %}
+{% include prev_next.md %}

data/docs/_docs/upgrading/upgrade4.md

@@ -3,7 +3,7 @@ title: Upgrading to Version 4.0
 short_title: Version 4.0
 order: 2
 categories: upgrading
-nav_order: 35
+nav_order: 36
 ---
 
 A major change in ufo from version 3 to 4 is that the ECS service is now created by CloudFormation. If you have an existing ECS service deployed by ufo version 3, when you deploy your app again with ufo version 4, there will be a new additional ECS service created. Here is the recommended upgrade path.
@@ -45,4 +45,4 @@ File | Changes
 .ufo/settings/cfn/default.yml | Starter cfn settings file.
 .ufo/settings/network/default.yml | This generated file will have the vpc and subnets that you specified above.  You can change them directly in this file to control what network settings ufo uses.
 
-{% include prev_next.md %}
+{% include prev_next.md %}

data/docs/_docs/variables.md

@@ -52,4 +52,4 @@ When `ufo ship` is ran with `UFO_ENV=development` the `variables/development.rb`
 ])
 ```
 
-{% include prev_next.md %}
+{% include prev_next.md %}

data/docs/_includes/subnav.html

@@ -26,6 +26,7 @@
                 <li><a href="{% link _docs/variables.md %}">Shared Variables</a></li>
                 <li><a href="{% link _docs/helpers.md %}">Helpers</a></li>
                 <li><a href="{% link _docs/conventions.md %}">Conventions</a></li>
+                <li><a href="{% link _docs/ufo-logs.md %}">Ufo Logs</a></li>
                 <li><a href="{% link _docs/ufo-env.md %}">Ufo Env</a></li>
                 <li><a href="{% link _docs/ufo-env-extra.md %}">Ufo Env Extra</a></li>
                 <li><a href="{% link _docs/ufo-current.md %}">Ufo Current</a></li>

data/docs/_reference/ufo-logs.md

@@ -0,0 +1,40 @@
+---
+title: ufo logs
+reference: true
+---
+
+## Usage
+
+    ufo logs
+
+## Description
+
+Prints out logs
+
+## Examples
+
+    ufo logs demo-web
+
+If you have a current service name set.
+
+    ufo current --service demo-web
+    ufo logs # follow by default
+    ufo logs --no-follow
+    ufo logs --format detailed # to show stream too
+
+
+## Options
+
+```
+[--follow], [--no-follow]          #  Whether to continuously poll for new logs. To exit from this mode, use Control-C.
+                                   # Default: true
+[--since=SINCE]                    # From what time to begin displaying logs.  By default, logs will be displayed starting from 1 minutes in the past. The value provided can be an ISO 8601 timestamp or a relative time.
+[--format=FORMAT]                  # The format to display the logs. IE: detailed or short.  With detailed, the log stream name is also shown.
+                                   # Default: simple
+[--filter-pattern=FILTER_PATTERN]  # The filter pattern to use. If not provided, all the events are matched
+[--verbose], [--no-verbose]        
+[--mute], [--no-mute]              
+[--noop], [--no-noop]              
+[--cluster=CLUSTER]                # Cluster.  Overrides .ufo/settings.yml.
+```
+

data/docs/articles.md

@@ -1,6 +1,6 @@
 ---
 title: Articles
-nav_order: 45
+nav_order: 46
 ---
 
 * [How to Create Unlimited Extra Environments
@@ -11,4 +11,4 @@ nav_order: 45
 * [Heroku vs ECS Fargate vs EC2 On-Demand vs EC2 Spot Pricing Comparison](https://blog.boltops.com/2018/04/22/heroku-vs-ecs-fargate-vs-ec2-on-demand-vs-ec2-spot-pricing-comparison)
 * [Ufo Ship on AWS ECS Presentation](http://www.slideshare.net/tongueroo/ufo-ship-for-aws-ecs-70885296)
 
-{% include prev_next.md %}
+{% include prev_next.md %}

data/docs/docs.md

@@ -26,4 +26,4 @@ You might like these articles:
 
 Also, the [UFO Tutorial Docs]({% link _docs/tutorial.md %}) provide a detail walkthrough on how each UFO step works.
 
-{% include prev_next.md %}
+{% include prev_next.md %}

data/docs/quick-start.md

@@ -129,4 +129,4 @@ Here's an article that compares the cost of ECS Fargate: [Heroku vs ECS Fargate
 
 {% include repo_push_access.md %}
 
-{% include prev_next.md %}
+{% include prev_next.md %}

data/docs/reference.md

@@ -12,6 +12,7 @@ title: CLI Reference
 * [ufo destroy]({% link _reference/ufo-destroy.md %})
 * [ufo docker]({% link _reference/ufo-docker.md %})
 * [ufo init]({% link _reference/ufo-init.md %})
+* [ufo logs]({% link _reference/ufo-logs.md %})
 * [ufo network]({% link _reference/ufo-network.md %})
 * [ufo ps]({% link _reference/ufo-ps.md %})
 * [ufo releases]({% link _reference/ufo-releases.md %})

data/lib/ufo/aws_service.rb

@@ -5,6 +5,8 @@ require "aws-sdk-ecr"
 require "aws-sdk-ecs"
 require "aws-sdk-elasticloadbalancingv2"
 
+require "aws_mfa_secure/ext/aws" # add MFA support
+
 module Ufo
   module AwsService
     def cloudformation

data/lib/ufo/cli.rb

@@ -187,6 +187,16 @@ module Ufo
       Status.new(service, options).run
     end
 
+    desc "logs", "Prints out logs"
+    long_desc Help.text(:logs)
+    option :follow, default: true, type: :boolean, desc: " Whether to continuously poll for new logs. To exit from this mode, use Control-C."
+    option :since, desc: "From what time to begin displaying logs.  By default, logs will be displayed starting from 1 minutes in the past. The value provided can be an ISO 8601 timestamp or a relative time."
+    option :format, default: "simple", desc: "The format to display the logs. IE: detailed or short.  With detailed, the log stream name is also shown."
+    option :filter_pattern, desc: "The filter pattern to use. If not provided, all the events are matched"
+    def logs(service=:current)
+      Logs.new(service, options).run
+    end
+
     desc "completion *PARAMS", "Prints words for auto-completion."
     long_desc Help.text("completion")
     def completion(*params)

data/lib/ufo/core.rb

@@ -9,7 +9,7 @@ module Ufo
       task_definition_path = "#{Ufo.root}/.ufo/output/#{task_definition}.json"
       unless File.exist?(task_definition_path)
         puts "ERROR: Unable to find the task definition at #{task_definition_path}.".color(:red)
-        puts "Are you sure you have defined it in ufo/template_definitions.rb and it has been generated correctly in .ufo/output?".color(:red)
+        puts "Are you sure you have defined it in .ufo/task_definitions.rb and it has been generated correctly in .ufo/output?".color(:red)
         puts "If you are calling `ufo deploy` directly, you might want to generate the task definition first with `ufo tasks build`."
         exit 1
       end
@@ -42,10 +42,10 @@ module Ufo
       return if ENV['TEST']
       return unless File.exist?("#{Ufo.root}/.ufo/settings.yml") # for rake docs
       return unless settings # Only load if within Ufo project and there's a settings.yml
-      data = settings[Ufo.env] || {}
-      if data["aws_profile"]
-        puts "Using AWS_PROFILE=#{data["aws_profile"]} from UFO_ENV=#{Ufo.env} in config/settings.yml"
-        ENV['AWS_PROFILE'] = data["aws_profile"]
+      data = settings || {}
+      if data[:aws_profile]
+        puts "Using AWS_PROFILE=#{data[:aws_profile]} from UFO_ENV=#{Ufo.env} in config/settings.yml"
+        ENV['AWS_PROFILE'] = data[:aws_profile]
       end
     end
 

data/lib/ufo/dsl/helper.rb

@@ -30,7 +30,9 @@ module Ufo
       def env_vars(text)
         lines = filtered_lines(text)
         lines.map do |line|
-          key,*value = line.strip.split("=").map {|x| x.strip}
+          key,*value = line.strip.split("=").map do |x|
+            remove_surrounding_quotes(x.strip)
+          end
           value = value.join('=')
           {
             name: key,
@@ -39,6 +41,16 @@ module Ufo
         end
       end
 
+      def remove_surrounding_quotes(s)
+        if s =~ /^"/ && s =~ /"$/
+          s.sub(/^["]/, '').gsub(/["]$/,'') # remove surrounding double quotes
+        elsif s =~ /^'/ && s =~ /'$/
+          s.sub(/^[']/, '').gsub(/[']$/,'') # remove surrounding single quotes
+        else
+          s
+        end
+      end
+
       def filtered_lines(text)
         lines = text.split("\n")
         # remove comment at the end of the line

data/lib/ufo/help/logs.md

@@ -0,0 +1,10 @@
+## Examples
+
+    ufo logs demo-web
+
+If you have a current service name set.
+
+    ufo current --service demo-web
+    ufo logs # follow by default
+    ufo logs --no-follow
+    ufo logs --format detailed # to show stream too

data/lib/ufo/logs.rb

@@ -0,0 +1,56 @@
+require "aws-logs"
+
+module Ufo
+  class Logs < Base
+    include AwsService
+
+    delegate :service, to: :info
+
+    def run
+      log = find_log_group_name
+      puts "Showing logs for log group: #{log["awslogs-group"]} and stream prefix #{log["awslogs-stream-prefix"]}"
+      if log
+        cloudwatch_tail(log)
+      else
+        puts "Unable to find log group for service: #{service.service_name}"
+      end
+    end
+
+    def find_log_group_name
+      resp = ecs.describe_task_definition(task_definition: info.service.task_definition)
+
+      container_definitions = resp.task_definition.container_definitions
+
+      unless container_definitions.size == 1
+        puts "ERROR: ufo logs command only supports 1 container definition in the ECS task definition".color(:red)
+        return
+      end
+
+      definition = container_definitions.first
+      log_conf = definition.log_configuration
+
+      if log_conf && log_conf.log_driver == "awslogs"
+        # options["awslogs-group"]
+        # options["awslogs-region"]
+        # options["awslogs-stream-prefix"]
+        log_conf.options
+      else
+        puts "Only supports awslogs driver. Detected log_driver: #{log_conf.log_driver}"
+        return
+      end
+    end
+
+    def cloudwatch_tail(log={})
+      o = {
+        log_group_name: log["awslogs-group"],
+        log_stream_name_prefix: log["awslogs-stream-prefix"],
+        since: @options[:since] || "10m", # by default, search only 10 mins in the past
+        follow: @options[:follow],
+        format: @options[:format],
+      }
+      o[:filter_pattern] = @options[:filter_pattern] if @options[:filter_pattern]
+      cw_tail = AwsLogs::Tail.new(o)
+      cw_tail.run
+    end
+  end
+end

data/lib/ufo/tasks/register.rb

@@ -70,15 +70,18 @@ module Ufo
 
       definitions = data[:container_definitions]
       definitions.each_with_index do |definition, i|
-        next unless definition[:log_configuration]
-        options = definition[:log_configuration][:options]
-        next unless options
+        next unless definition[:log_configuration] || definition[:firelens_configuration]
+        { log_configuration: 'logConfiguration',
+          firelens_configuration: 'firelensConfiguration' }.each_pair do |key, value|
 
-        # LogConfiguration options do not get transformed and keep their original
-        # structure:
-        #   https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/ECS/Types/ContainerDefinition.html
-        original_definition = original_data["containerDefinitions"][i]
-        definition[:log_configuration][:options] = original_definition["logConfiguration"]["options"]
+          next unless definition.dig(key, :options)
+
+          # LogConfiguration and firelensConfiguration options do not get transformed and
+          # keep their original structure:
+          #   https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/ECS/Types/ContainerDefinition.html
+          original_definition = original_data["containerDefinitions"][i]
+          definition[key][:options] = original_definition[value]["options"]
+        end
       end
 
       data

data/lib/ufo/version.rb

@@ -1,3 +1,3 @@
 module Ufo
-  VERSION = "4.5.8"
+  VERSION = "4.6.1"
 end

data/spec/fixtures/mocks/logs/awslogs.json

@@ -0,0 +1,70 @@
+{
+  "task_definition": {
+    "task_definition_arn": "arn:aws:ecs:us-west-2:112233445566:task-definition/demo-web:1",
+    "container_definitions": [
+      {
+        "name": "web",
+        "image": "112233445566.dkr.ecr.us-west-2.amazonaws.com/demo/sinatra:ufo-2020-01-16T22-56-26-a28378e80",
+        "cpu": 256,
+        "memory": 1024,
+        "memory_reservation": 1024,
+        "port_mappings": [
+          {
+            "container_port": 4567,
+            "host_port": 4567,
+            "protocol": "tcp"
+          }
+        ],
+        "essential": true,
+        "command": [
+          "bin/web"
+        ],
+        "environment": [],
+        "log_configuration": {
+          "log_driver": "awslogs",
+          "options": {
+            "awslogs-group": "ecs/demo-web",
+            "awslogs-region": "us-west-2",
+            "awslogs-stream-prefix": "demo"
+          }
+        }
+      }
+    ],
+    "family": "demo-web",
+    "task_role_arn": "arn:aws:iam::112233445566:role/demo",
+    "network_mode": "awsvpc",
+    "revision": 1,
+    "volumes": [
+    ],
+    "status": "ACTIVE",
+    "requires_attributes": [
+      {
+        "name": "com.amazonaws.ecs.capability.logging-driver.awslogs"
+      },
+      {
+        "name": "com.amazonaws.ecs.capability.ecr-auth"
+      },
+      {
+        "name": "com.amazonaws.ecs.capability.docker-remote-api.1.19"
+      },
+      {
+        "name": "com.amazonaws.ecs.capability.docker-remote-api.1.21"
+      },
+      {
+        "name": "com.amazonaws.ecs.capability.task-iam-role"
+      },
+      {
+        "name": "com.amazonaws.ecs.capability.docker-remote-api.1.18"
+      },
+      {
+        "name": "ecs.capability.task-eni"
+      }
+    ],
+    "placement_constraints": [
+
+    ],
+    "compatibilities": [
+      "EC2"
+    ]
+  }
+}