ufo 4.5.4 → 4.5.9

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 66df80258881426de197903fb8f5fb1e6105d688f2cde690b98884f060ea134b
-  data.tar.gz: c4032f3c68551d0ba3436953ee001b71fa5a629d62d8684b01a9d2b1f170d87a
+  metadata.gz: b4c0e7758496f288d64dad5ba85245f82805645950e9af2aceaf098d3eb0b62a
+  data.tar.gz: e0ea316e243b735c20e64f52de6a26f44b0ab3ee6ae38f807ef0d4cbe123cf80
 SHA512:
-  metadata.gz: def48872244ef871377f433b4bb56fdea3bcf0a1e9475a714439b2f566af64644d0eee8f2b81a4c0b54049467db24cd3ae1e37d99d01f85e2a4c64a5c8061d8d
-  data.tar.gz: 686dcad5e51f5b9d62308e44dc1c9ebec202c4607389564d2764a6b37ddb56cdff776e8fef8e87f421fdd532865f4ca1dced624e36158ed273027e7b63e0d7ec
+  metadata.gz: a45969245092d9b836c04be93e17d2a30057b3e34a3afaf3763dbe8c437c16038d805d34198077927818cd6bdda47e4dcd480a624eba5939f6299ff717455748
+  data.tar.gz: a434c5c2d6af48700399e93a40db5e1641135ccd22b96ec31dbd2da668e52f102f6923676db545679330963acf7c565ea594b8053424a15222ef86baba86790c

data/CHANGELOG.md

@@ -3,9 +3,27 @@
 All notable changes to this project will be documented in this file.
 This project *tries* to adhere to [Semantic Versioning](http://semver.org/), even before v1.0.
 
+## [4.5.9]
+- fix ufo_env aws_profile tight binding
+
+## [4.5.8]
+- #91 added helper scripts to dianose and resolve the SSL issues - added docs to help explain and save the user time and research
+- improve cancel command
+- update /up check starter example
+
+## [4.5.7]
+- #88 update starter variables template with += example
+
+## [4.5.6]
+- fix outgoing egress rule to allow ping
+
+## [4.5.5]
+- adjust default health check thresholds in skeleton
+- improve error handling for UPDATE\_ROLLBACK\_FAILED state
+
 ## [4.5.4]
-- Merge pull request #85 `ufo docker compile` command. ability to compile Dockerfile from Dockerfile.erb w/o building
-- Merge pull request #86 slight improvement to `ufo docker compile`
+- #85 `ufo docker compile` command. ability to compile Dockerfile from Dockerfile.erb w/o building
+- #86 slight improvement to `ufo docker compile`
 
 ## [4.5.3]
 - fix error exit code when unable to find task definition

data/README.md

@@ -20,7 +20,7 @@ See [ufoships.com](http://ufoships.com) for full documentation.
 
 ## Important
 
-If you are on version 3, you can run `ufo upgrade v3to4` within your project to upgrade it to version 4.  Refer to the [CHANGELOG](CHANGELOG.md) and the [Upgrade 4 Docs](http://ufoships.com/docs/upgrade4/).
+If you are upgrading, please refer to the [Upgrading docs](https://ufoships.com/docs/upgrading/)
 
 ## Installation
 
@@ -42,7 +42,7 @@ Congratulations, you have successfully used ufo to deploy to an ECS service.
 
 ## Load Balancer Support
 
-Ufo can also create a load balancer as part of creating the ECS service if you wish. Underneath the hood, ufo uses CloudFormation to create the load balancer.  More information can be found at the [load balancer support docs](http://ufoships.com/docs/load-balancer/).
+Ufo can also create a load balancer as part of creating the ECS service if you wish. Underneath the hood, ufo uses CloudFormation to create the load balancer.  More information can be found at the [load balancer support docs](https://ufoships.com/docs/extras/load-balancer/).
 
 ## Articles
 

data/docs/_docs/extras/ecs-network-mode.md

@@ -3,15 +3,25 @@ title: ECS Network Mode
 nav_order: 26
 ---
 
-## Pros and Cons: awsvpc vs bridge network mode
+## Pros and Cons: bridge network mode
 
 With network bridge mode, the Docker containers of multiple services share the EC2 container instance's security group. So you have less granular control over opening ports for specific services only. For example, let’s say service A and B both are configured use bridge network mode. If you open up port 3000 for service A, it will also open up port 3000 for service B because they use the same security group at the EC2 instance level.
 
 One advantage of bridge mode is you can use dynamic port mapping and do not have to worry about network card limits.
 
-With awsvpc network mode, you must consider the limit of ethernet cards for the instance type. The table that lists the limits are under section the aws EC2 docs under [IP Addresses Per Network Interface Per Instance Type](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html) For example, a t2.large instance has a limit of 3 ethernet cards. This means, at most, you can run 3 ECS tasks on that instance in awsvpc network mode.  The network card limit ranges from 3 to 15 ethernet cards depending on the instance type.
+## Pros and Cons: awsvpc mode
 
-The advantage of awsvpc mode is that since the ECS task has its own network card and security group, there’s more granular control of the permissions per ECS service. For example, when service A and B are using awsvpc mode, they can have different security groups associated with them. In this mode, ufo creates a security group and sets up the permissions so the load balancer can talk to the containers.  You can also add additional security group to the `.ufo/settings/network/default.yml` config.
+With awsvpc network mode, you must consider the limit of ethernet cards for the instance type. If the instance supports ENI Trunking, then this is limit is decently large. However, if the instance does not support ENI Trunking, then the ENI limit is rather small.
+
+For ENI Trunking Task limits per instance: [Elastic Network Interface Trunking](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/container-instance-eni.html)
+
+For example, a m5.large instance has a limit of 10 tasks per instance.
+For EC2 instances that do not support ENI Trunking,
+the table that lists the limits are under section the aws EC2 docs under [IP Addresses Per Network Interface Per Instance Type](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html)
+
+For example, a t3.small instance has a limit of 3 ethernet cards. This means, at most, you can run 2 ECS tasks on that instance in awsvpc network mode, since one network card is already used by the host.
+
+In awsvpc mode, each ECS task gets its own network card. The advantage is there’s more granular control of the permissions per ECS service. For example, when service A and B are using awsvpc mode, they can have different security groups associated with them. In this mode, ufo creates a security group and sets up the permissions so the load balancer can talk to the containers.  You can also add additional security groups to the `.ufo/settings/network/default.yml` config.
 
 The following table summarizes the pros and cons:
 
@@ -20,4 +30,8 @@ Network mode | Pros | Cons
 bridge | The numbers of containers you can run will not be limited due to EC2 instance network cards limits. | Less fine grain security control over security group permissions with multiple ECS services.
 awsvpc | Fine grain security group permissions for each ECS service. | The number of containers can be limited by the number of network cards the EC2 instance type supports.
 
+## Recommendation
+
+It is generally recommended to use awsvpc mode with ENI trunking supported instances. You get the best of both worlds in this situation: a strong security posture as well as container density.
+
 {% include prev_next.md %}

data/docs/_docs/ssl_errors.md

@@ -0,0 +1,41 @@
+---
+Title: SSL Errors
+# nav_order:
+---
+
+UFO uses the AWS Ruby SDK and the underlying default SSL certificate chain configured in your active Ruby and
+OpenSSL to communicate to your AWS environment. This means that you _must correctly configure_ your Ruby and OpenSSL to have all the needed ROOT certificates for UFO to be able to communicate to AWS - _especially_ if you are behind a proxy or a corporate SSL-Proxy.
+
+If you are behind a corporate SSL proxy and you have not updated system, OpenSSL and Ruby certificate chains to include the needed corporate root certificates, you will see errors, such as:
+
+```
+Seahorse::Client::NetworkingError: SSL_connect returned=1 errno=0 state=error: certificate verify failed (self signed certificate in certificate chain)
+  ~/.rbenv/versions/2.6.0/lib/ruby/2.6.0/net/protocol.rb:44:in `connect_nonblock'
+  ~/.rbenv/versions/2.6.0/lib/ruby/2.6.0/net/protocol.rb:44:in `ssl_socket_connect'
+  ~/.rbenv/versions/2.6.0/lib/ruby/2.6.0/net/http.rb:996:in `connect'
+  ~/.rbenv/versions/2.6.0/lib/ruby/2.6.0/net/http.rb:930:in `do_start'
+  ~/.rbenv/versions/2.6.0/lib/ruby/2.6.0/net/http.rb:925:in `start'
+```
+
+## Helper Scripts
+
+The `docs/utils` directory has a few scripts that should be able to help you resolve these issues and track down which certs are giving you problems.
+
+- `ssl-doctor.rb` is from the very useful examples at <https://github.com/mislav/ssl-tools>, and it can help you find the missing ROOT cert in your certificate chain and give suggestion on getting OpenSSL working correctly.
+- `update-cert-chains.sh` will help you update your Ruby and OpenSSL chains by adding in the missing ROOT cert and also pulling in the OSX System Root to your rbenv environment.
+- `test-aws-api-access.rb` should now return a list of the S3 buckets for the current AWS profile that is active.
+
+## Trouble-shooting
+
+### Update Brew and OpenSSL
+
+- `brew update`
+- `brew upgrade openssl`
+
+### Use the Helper Scripts to find the trouble spot
+
+Once you have updated OpenSSL and your `brew` packages, use the helper scripts above to see if you can track down the missing certificate in your certificate chain.
+
+The `update-cert-chain.sh` file was created using the suggestions from <https://gemfury.com/help/could-not-verify-ssl-certificate/>. Please review the information at <https://gemfury.com/help/could-not-verify-ssl-certificate/> if the `Helper Scripts` above do not fully resolve your issue.
+
+The `test-aws-api-access.rb` uses examples from the <https://docs.aws.amazon.com/sdk-for-ruby/v3/developer-guide/quick-start-guide.html> for using and configuring the Ruby AWS SDK on your system.

data/docs/_docs/ufo-env-extra.md

@@ -3,6 +3,8 @@ title: UFO_ENV_EXTRA
 nav_order: 21
 ---
 
+<div class="video-box"><div class="video-container"><iframe src="https://www.youtube.com/embed/UVQuwQGToYE" frameborder="0" allowfullscreen=""></iframe></div></div>
+
 Ufo has an concept of extra environments. This is controlled by the `UFO_ENV_EXTRA` variable.  By setting `UFO_ENV_EXTRA` you can create additional identical ECS services or environments.
 
     ufo ship demo-web # creates a demo-web ecs service

data/docs/_includes/subnav.html

@@ -58,7 +58,7 @@
                 <li><a href="{% link _docs/more/why-cloudformation.md %}">Why CloudFormation</a></li>
                 <li><a href="{% link _docs/more/customize-cloudformation.md %}">Customize CloudFormation</a></li>
                 <li><a href="{% link _docs/more/stuck-cloudformation.md %}">Stuck CloudFormation</a></li>
-                <li><a href="{% link _docs/more/run-in-pieces.md %}">Run In Pieces</a></li>
+                <li><a href="{% link _docs/more/run-in-pieces.md %}">Run In Steps</a></li>
                 <li><a href="{% link _docs/more/single-task.md %}">Run Single Task</a></li>
                 <li><a href="{% link _docs/more/migrations.md %}">Database Migrations</a></li>
                 <li><a href="{% link _docs/more/automated-cleanup.md %}">Automated Cleanup</a></li>

data/docs/utils/ssl-doctor.rb

@@ -0,0 +1,89 @@
+# Usage: ruby doctor.rb [HOST=status.github.com[:PORT=443]]
+# see: https://github.com/mislav/ssl-tools
+require 'rbconfig'
+require 'net/https'
+
+if ARGV[0] =~ /^[^-]/
+  host, port = ARGV[0].split(':', 2)
+else
+  host = 'status.github.com'
+end
+port ||= 443
+
+ruby = File.join(RbConfig::CONFIG['bindir'], RbConfig::CONFIG['ruby_install_name'])
+ruby_version = RUBY_VERSION
+if patch = RbConfig::CONFIG['PATCHLEVEL']
+  ruby_version += "-p#{patch}"
+end
+puts "%s (%s)" % [ruby, ruby_version]
+
+openssl_dir = OpenSSL::X509::DEFAULT_CERT_AREA
+mac_openssl = '/System/Library/OpenSSL' == openssl_dir
+puts "%s: %s" % [OpenSSL::OPENSSL_VERSION, openssl_dir]
+[OpenSSL::X509::DEFAULT_CERT_DIR_ENV, OpenSSL::X509::DEFAULT_CERT_FILE_ENV].each do |key|
+  puts "%s=%s" % [key, ENV[key].to_s.inspect]
+end
+
+ca_file = ENV[OpenSSL::X509::DEFAULT_CERT_FILE_ENV] || OpenSSL::X509::DEFAULT_CERT_FILE
+ca_path = (ENV[OpenSSL::X509::DEFAULT_CERT_DIR_ENV] || OpenSSL::X509::DEFAULT_CERT_DIR).chomp('/')
+
+puts "\nHEAD https://#{host}:#{port}"
+http = Net::HTTP.new(host, port)
+http.use_ssl = true
+
+# Explicitly setting cert_store like this is not needed in most cases but it
+# seems necessary in edge cases such as when using `verify_callback` in some
+# combination of Ruby + OpenSSL versions.
+http.cert_store = OpenSSL::X509::Store.new
+http.cert_store.set_default_paths
+
+http.verify_mode = OpenSSL::SSL::VERIFY_PEER
+failed_cert = failed_cert_reason = nil
+
+if mac_openssl
+  warn "warning: will not be able show failed certificate info on OS X's OpenSSL"
+  # This drives me absolutely nuts. It seems that on Rubies compiled against OS X's
+  # system OpenSSL, the mere fact of defining a `verify_callback` makes the
+  # cert verification fail for requests that would otherwise be successful.
+else
+  http.verify_callback = lambda { |verify_ok, store_context|
+    if !verify_ok
+      failed_cert = store_context.current_cert
+      failed_cert_reason = "%d: %s" % [ store_context.error, store_context.error_string ]
+    end
+    verify_ok
+  }
+end
+
+user_agent = "net/http #{ruby_version}"
+req = Net::HTTP::Head.new('/', 'user-agent' => user_agent)
+
+begin
+  res = http.start { http.request(req) }
+  abort res.inspect if res.code.to_i >= 500
+  puts "OK"
+rescue Errno::ECONNREFUSED
+  puts "Error: connection refused"
+  exit 1
+rescue OpenSSL::SSL::SSLError => e
+  puts "#{e.class}: #{e.message}"
+
+  if failed_cert
+    puts "\nThe server presented a certificate that could not be verified:"
+    puts "  subject: #{failed_cert.subject}"
+    puts "  issuer: #{failed_cert.issuer}"
+    puts "  error code %s" % failed_cert_reason
+  end
+
+  ca_file_missing = !File.exist?(ca_file) && !mac_openssl
+  ca_path_empty = Dir["#{ca_path}/*"].empty?
+
+  if ca_file_missing || ca_path_empty
+    puts "\nPossible causes:"
+    puts "  `%s' does not exist" % ca_file if ca_file_missing
+    puts "  `%s/' is empty" % ca_path if ca_path_empty
+  end
+
+  exit 1
+end
+

data/docs/utils/test-aws-api-access.rb

@@ -0,0 +1,11 @@
+# usage 'ruby s3-cert-chain-test.rb'
+# see: https://docs.aws.amazon.com/sdk-for-ruby/v3/developer-guide/quick-start-guide.html
+
+require 'aws-sdk-s3' # v2: require 'aws-sdk'
+#Aws.use_bundled_cert!
+
+s3 = Aws::S3::Resource.new(region: 'us-east-1')
+
+s3.buckets.limit(50).each do |b|
+  puts "#{b.name}"
+end

data/docs/utils/update-cert-chains.sh

@@ -0,0 +1,11 @@
+#!/bin/bash
+
+cert_file=$(ruby -ropenssl -e 'puts OpenSSL::X509::DEFAULT_CERT_FILE' 2>/dev/null)
+echo 'What is the uri to your organizations root certificate chain?'
+read -p 'org_root_chain: ' org_root_chain
+echo "$org_root_chain"
+curl "$org_root_chain" -o org_chain.txt
+cat org_chain.txt >> "$cert_file"
+mkdir -p "${cert_file%/*}"
+security find-certificate -a -p /Library/Keychains/System.keychain > "$cert_file"
+security find-certificate -a -p /System/Library/Keychains/SystemRootCertificates.keychain >> "$cert_file"

data/lib/cfn/stack.yml

@@ -237,9 +237,7 @@ Resources:
       # Outbound access: instance needs access to internet to pull down image
       # or else get CannotPullContainerError
       SecurityGroupEgress:
-      - IpProtocol: tcp
-        FromPort: '0'
-        ToPort: '65535'
+      - IpProtocol: "-1"
         CidrIp: 0.0.0.0/0
         Description: outbound traffic
       Tags:

data/lib/template/.ufo/settings/cfn/default.yml.tt

@@ -18,11 +18,12 @@ target_group:
                   #   network elb: TCP
                   # so we can keep this commented out, unless we need HTTPS at the app level
   # Health check settings are supported by application load balancer only:
-  # health_check_path: /upcheck
-  # health_check_interval_seconds: 30 # 10 or 30 for network ELB
+  # health_check_path: /up # health check
+  health_check_interval_seconds: 10 # default: 30. Network ELB can only take 10 or 30
+  healthy_threshold_count: 2
+  unhealthy_threshold_count: 2 # default: 10
   # health_check_protocol: HTTP # HTTP or HTTPS
   # health_check_port: traffic-port
-  # unhealthy_threshold_count: 10
   target_group_attributes:
   - key: deregistration_delay.timeout_seconds
     value: 10

data/lib/template/.ufo/variables/development.rb

@@ -1,7 +1,8 @@
 # Example ufo/variables/development.rb
 # More info on how variables work: http://ufoships.com/docs/variables/
 @cpu = 256
-@environment = helper.env_vars(%Q[
+# Refer to https://github.com/tongueroo/ufo/issues/87 as to why the += is used
+@environment += helper.env_vars(%Q[
   RAILS_ENV=development
   SECRET_KEY_BASE=secret
 ])

data/lib/template/.ufo/variables/production.rb

@@ -1,7 +1,7 @@
 # Example ufo/variables/production.rb
 # More info on how variables work: http://ufoships.com/docs/variables/
 @cpu = 256
-@environment = helper.env_vars(%Q[
+@environment += helper.env_vars(%Q[
   RAILS_ENV=production
   SECRET_KEY_BASE=secret
 ])

data/lib/ufo/cancel.rb

@@ -12,7 +12,7 @@ module Ufo
       if stack.stack_status == "CREATE_IN_PROGRESS"
         cloudformation.delete_stack(stack_name: @stack_name)
         puts "Canceling stack creation."
-      elsif stack.stack_status =~ /_IN_PROGRESS$/
+      elsif stack.stack_status == "UPDATE_IN_PROGRESS"
         cloudformation.cancel_update_stack(stack_name: @stack_name)
         puts "Canceling stack update."
       else

data/lib/ufo/core.rb

@@ -36,16 +36,16 @@ module Ufo
     end
     memoize :env_extra
 
-    # Overrides AWS_PROFILE based on the Ufo.env if set in configs/settings.yml
+    # Overrides AWS_PROFILE based on the Ufo.env if set in .ufo/settings.yml
     # 2-way binding.
     def set_aws_profile!
       return if ENV['TEST']
       return unless File.exist?("#{Ufo.root}/.ufo/settings.yml") # for rake docs
       return unless settings # Only load if within Ufo project and there's a settings.yml
-      data = settings[Ufo.env] || {}
-      if data["aws_profile"]
-        puts "Using AWS_PROFILE=#{data["aws_profile"]} from UFO_ENV=#{Ufo.env} in config/settings.yml"
-        ENV['AWS_PROFILE'] = data["aws_profile"]
+      data = settings || {}
+      if data[:aws_profile]
+        puts "Using AWS_PROFILE=#{data[:aws_profile]} from UFO_ENV=#{Ufo.env} in config/settings.yml"
+        ENV['AWS_PROFILE'] = data[:aws_profile]
       end
     end
 

data/lib/ufo/stack.rb

@@ -185,7 +185,11 @@ module Ufo
     def handle_stack_error(e)
       case e.message
       when /state and can not be updated/
-        puts "The #{@stack_name} stack is in #{"ROLLBACK_COMPLETE".color(:red)} and cannot be updated. Deleted the stack and try again."
+        puts "The #{@stack_name} stack is in a state that cannot be updated. Deleted the stack and try again."
+        puts "ERROR: #{e.message}"
+        if message.include?('UPDATE_ROLLBACK_FAILED')
+          puts "You might be able to do a 'Continue Update Rollback' and skip some resources to get the stack back into a good state."
+        end
         region = `aws configure get region`.strip rescue 'us-east-1'
         url = "https://console.aws.amazon.com/cloudformation/home?region=#{region}"
         puts "Here's the CloudFormation console url: #{url}"
@@ -203,7 +207,7 @@ module Ufo
     end
 
     def updatable?(stack)
-      stack.stack_status =~ /_COMPLETE$/
+      stack.stack_status =~ /_COMPLETE$/ || stack.stack_status == 'UPDATE_ROLLBACK_FAILED'
     end
   end
 end

data/lib/ufo/version.rb

@@ -1,3 +1,3 @@
 module Ufo
-  VERSION = "4.5.4"
+  VERSION = "4.5.9"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: ufo
 version: !ruby/object:Gem::Version
-  version: 4.5.4
+  version: 4.5.9
 platform: ruby
 authors:
 - Tung Nguyen
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2019-08-12 00:00:00.000000000 Z
+date: 2019-10-23 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: aws-sdk-cloudformation
@@ -333,6 +333,7 @@ files:
 - docs/_docs/settings/cfn.md
 - docs/_docs/settings/cluster.md
 - docs/_docs/settings/network.md
+- docs/_docs/ssl_errors.md
 - docs/_docs/structure.md
 - docs/_docs/tutorial-ufo-docker-build.md
 - docs/_docs/tutorial-ufo-init.md
@@ -459,6 +460,9 @@ files:
 - docs/quick-start.md
 - docs/reference.md
 - docs/style.css
+- docs/utils/ssl-doctor.rb
+- docs/utils/test-aws-api-access.rb
+- docs/utils/update-cert-chains.sh
 - exe/ufo
 - lib/cfn/stack.yml
 - lib/template/.env
@@ -622,7 +626,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.0.3
+rubygems_version: 3.0.6
 signing_key: 
 specification_version: 4
 summary: AWS ECS Deploy Tool