aws-cft-tools 0.1.0 → 0.1.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 32c8025214a49ca0ed274d45c3c6ee266dcbfb71
-  data.tar.gz: 135e9ef53e1081cc471bf095dd3a5e218364162e
+  metadata.gz: e2c4148f5b7aa3cf1465fcc735a1b3ebd8b559d9
+  data.tar.gz: 654d8ffb95b10000a1e3d364803fc014aaf2788a
 SHA512:
-  metadata.gz: bae7aae8cdc869ed5fcf53e9dd8e8419765d6ed475f8035ed482fbbdb9042a185df1323639a2263ef52e0169d0607a55ba11fe473eb0667a6327e4f28f016579
-  data.tar.gz: 8f06dceea86f02f3be1b3507a09da24d257484160437eb9d109c044965e3652084937bd6eadff37e7f64da92bf2534520ccc52a7aa17fdd599d2c560e71b9715
+  metadata.gz: 31d7266d177d7cc7359b35ddea6271b4c4f1d14ef8e4fdf850056e6a7823ffc4bef9ee802c85513e99944bab14110098c33e568ca1faa58883a41ae44efc6e60
+  data.tar.gz: 12d6742f6050e73e68a0a54d10138ba21d33fc3ac10be49e2b150ee248651b7be6f6337d34558e8a5ea51153e174b2a59e9487977e968897f74debb1652a5913

data/.rubocop.yml

@@ -1,6 +1,9 @@
 ---
 require: rubocop-rspec
 
+AllCops:
+  TargetRubyVersion: 2.4
+
 Metrics/BlockLength:
   Exclude:
     - 'aws-cft-tools.gemspec'

data/.travis.yml

@@ -3,3 +3,6 @@ language: ruby
 rvm:
   - 2.4.1
 before_install: gem install bundler -v 1.15.3
+script:
+  - bundle exec rake spec
+  - bundle exec rubocop

data/CHANGELOG.md

@@ -0,0 +1,23 @@
+# Changelog
+
+## [0.1.1] - 2017-12-15
+
+### Added
+
+* The `-d` and `--debug` options for `deploy` show more narrative. This can be useful if running parallel
+  jobs in a deployment (`-j` option with a value greater than one). This option is available for all
+  runbooks, but only the `deploy` runbook uses it in this release.
+* Documentation on why `aws-cft-tools` is better than plain vanilla CloudFormation or Terraform. (Issue #10)
+
+### Changed
+
+* Parallel deployments use threads. We weren't always handling output properly, so some output could be lost
+  if there was an error. We've reworked how we coordinate output from threads to reduce the chance of this
+  happening. (Issue #7)
+* Deployments special case the `-j1` option to avoid threads. If you run into issues with `-j` and a value
+  greater than one, run without the option or with `-j1` to make sure that threads aren't the source of the
+  problem. (Issue #7)
+
+## [0.1.0] - 2017-11-29
+
+Initial release.

data/README.md

@@ -1,9 +1,18 @@
 # AwsCftTools
 
+[![Gem Version](https://badge.fury.io/rb/aws-cft-tools.svg)](https://badge.fury.io/rb/aws-cft-tools)
+[![Build Status](https://travis-ci.org/USSBA/aws-cft-tools.svg?branch=master)](https://travis-ci.org/USSBA/aws-cft-tools)
+
 CloudFormation and related services provide a way to manage infrastructure state in "the cloud." This
 gem and its included command (`aws-cft`) build on top of this state management system to create an
 infrastructure management solution native to the AWS environment.
 
+`aws-cft-tools` empowers users to organize their CloudFormation templates using any form of directory
+structure, without the need to tediously deploy their templates in a specific order or create quickly
+outdated scripts to manage the deployment thereof.  This project links together templates using the
+Export/ImportValue features of CloudFormation to determine the order of operations, manages stack
+names, and supports multiple parallel "Environments" within a single AWS account.
+
 ## Installation
 
 Add this line to your application's Gemfile:
@@ -71,6 +80,62 @@ update the version number in `version.rb`, and then run `bundle exec rake releas
 tag for the version, push git commits and tags, and push the `.gem` file to
 [rubygems.org](https://rubygems.org).
 
+## Why `aws-cft-tools`?
+
+`aws-cft-tools` is designed to work in an "infrastructure as code" DevOps environment. Infrastructure is
+software that is developed, tested, peer reviewed, and finally merged and deployed.
+
+### "Vanilla" CloudFormation
+
+When first using CloudFormation, it is very easy to launch a single stack and get off the ground quickly.
+As you move forward, users quickly find out that their Templates need to be managed in source control.
+Later, users want to test their infrastructure changes in a different Environment, so a "dev" layer is
+created, then an "integration", then a "staging", etc.  Before too long, launching stacks is a nightmare
+due to dependency conflicts, manual naming failures of Stacks, typos, and so on.  On top of that,
+remembering which Stacks have been deployed for which environment becomes impossible, so infrastructure
+drift is inevitable.
+
+This tool builds on top of the normal progression of teams using CloudFormation, enabling managed
+Environments using parameters on templates.  It offers simple deployments to roll out a full stack in
+a new environment with a single command.  It allows developers to continue to use CloudFormation for all
+their infrastructure, while vastly simplifying the deployment and retraction process.
+
+### Ansible
+
+[Ansible](https://www.ansible.com/) provides features that are a mix of infrastructure management and
+instance configuration. For example, Ansible can do the work of TerraForm and Chef, combined. However,
+Ansible works best when working with an expected inventory of resources. It makes changes to bring
+infrastructure in line with the inventory. `aws-cft-tools` only manages CloudFormation templates and leaves
+configuration of instances to other tools such as Chef or Ansible.
+
+#### Using Ansible with `aws-cft-tools`
+
+Ansible can manage the production of an Amazon Machine Image (AMI). It can spin up a temporary EC2 instance
+and install all of the necessary system packages, make any configuration changes, and trigger the creation
+of a tagged AMI. If the AMI is tagged with an Environment and Role, then `aws-cft-tools` can discover the
+AMI and provide it as a parameter to any CloudFormation stacks that require the image. For example, creating
+a new AMI and then using `aws-cft-tools` to deploy the CloudFormation Template for an auto-scaling group
+that uses that AMI can result in the deployment of a new version of an application.
+
+### TerraForm
+
+[TerraForm](https://www.terraform.io/) and `aws-cft-tools` are solving similar problems with fundamentally
+different approaches. TerraForm is designed to work with multiple cloud providers while `aws-cft-tools` is
+specific to AWS. So TerraForm can't depend on features that aren't provided by all cloud providers. Thus,
+TerraForm requires a state file that introduces some complexity into managing infrastructure.
+
+Using `aws-cft-tools` doesn't mean infrastructure management is less complex than when using TerraForm. Only
+that the complexity is different. Instead of managing a state file outside of AWS, `aws-cft-tools` assumes
+that AWS is the source of all state information.
+
+Rather than computing changes, for example, `aws-cft-tools` requests a list of changes from AWS for a given
+change in template and parameters. This does take more time than if all of that information was in a local
+state file, but it ensures that any changes reflect the current deployment.
+
+In exchange for taking a little more time to make changes (e.g., pull requests and code reviews after
+initial development), teams can work on different parts of the infrastructure without having to coordinate
+with each other.
+
 ## Building Gem for Local Use
 
 ```shell

data/code.json

@@ -16,7 +16,7 @@
         "exemptionText": null
       },
       "vcs": "git",
-      "laborHours": 1,
+      "laborHours": 182,
       "tags": ["SBA", "DevOps", "AWS"],
       "contact": { "email": "Andrew.Davy@sba.gov" }
     }

data/exe/aws-cft

@@ -21,6 +21,7 @@ Clamp do
   option ['-T', '--tag'], 'NAME:VALUE', 'require a tag have the given value (may be given more than once)',
          multivalued: true
   option ['-v', '--[no-]verbose'], :flag, 'verbose narration of actions'
+  option ['-D', '--debug'], :flag, 'extra verbosity to aid in debugging'
   option '--version', :flag, 'Show version' do
     puts AwsCftTools::VERSION
     exit(0)
@@ -160,10 +161,16 @@ Clamp do
       profile: profile,
       root: root,
       region: region,
+      tags: tag_hash
+    }.merge(flag_options)
+  end
+
+  def flag_options
+    {
       noop: noop?,
       check: check?,
       verbose: verbose?,
-      tags: tag_hash
+      debug: debug?
     }
   end
 

data/lib/aws_cft_tools.rb

@@ -23,6 +23,7 @@ module AwsCftTools
   require 'aws_cft_tools/deletion_change'
   require 'aws_cft_tools/client'
   require 'aws_cft_tools/dependency_tree'
+  require 'aws_cft_tools/threaded_output'
   require 'aws_cft_tools/stack'
   require 'aws_cft_tools/template'
   require 'aws_cft_tools/template_set'

data/lib/aws_cft_tools/client/cft/changeset_management.rb

@@ -53,16 +53,36 @@ module AwsCftTools
           id = id_params(params)
 
           aws_client.create_change_set(params)
-
-          aws_client.wait_until(:change_set_create_complete, id)
-
+          return [] if wait_for_changeset(id) == :nochanges
           mapped_changes(AWSEnumerator.new(aws_client, :describe_change_set, id, &:changes).to_a)
-        rescue Aws::Waiters::Errors::FailureStateError
-          []
         ensure
           aws_client.delete_change_set(id)
         end
 
+        def wait_for_changeset(id, times_waited = 0)
+          times_waited += 1
+          aws_client.wait_until(:change_set_create_complete, id)
+        rescue Aws::Waiters::Errors::FailureStateError
+          status = check_failure(id)
+          return status unless status == :retry
+          raise_if_too_many_retries(params, times_waited)
+          sleep(2**times_waited + 1)
+          retry
+        end
+
+        def raise_if_too_many_retries(params, retries)
+          return if retries < 5
+          raise CloudFormationError, "Error waiting on changeset for #{params[:stack_name]}"
+        end
+
+        def check_failure(id)
+          status = aws_client.describe_change_set(id)
+          return :retry unless status.status == 'FAILED'
+          return :no_changes if status.status_reason.match?(/didn't contain changes/)
+          raise CloudFormationError,
+                "Error creating changeset for #{params[:stack_name]}: #{status.status_reason}"
+        end
+
         def id_params(params)
           {
             change_set_name: params[:change_set_name],

data/lib/aws_cft_tools/client/cft/stack_management.rb

@@ -25,7 +25,7 @@ module AwsCftTools
         def update_stack(template)
           aws_client.update_stack(update_stack_params(template))
           # we want to wait for the update to complete before we proceed
-          aws_client.wait_until(:stack_update_complete, stack_name: template.name)
+          wait_for_stack_operation(:stack_update_complete, template.name)
         rescue Aws::CloudFormation::Errors::ValidationError => exception
           raise exception unless exception.message.match?(/No updates/)
         end
@@ -39,7 +39,7 @@ module AwsCftTools
         def create_stack(template)
           aws_client.create_stack(create_stack_params(template))
           # we want to wait for the create to complete before we proceed
-          aws_client.wait_until(:stack_create_complete, stack_name: template.name)
+          wait_for_stack_operation(:stack_create_complete, template.name)
         end
 
         ##
@@ -55,6 +55,20 @@ module AwsCftTools
 
         private
 
+        def wait_for_stack_operation(op, stack_name, times_waited = 0)
+          times_waited += 1
+          aws_client.wait_until(op, stack_name: stack_name)
+        rescue Aws::Waiters::Errors::FailureStateError
+          raise_if_too_many_retries(stack_name, times_waited)
+          sleep(2**times_waited + 1)
+          retry
+        end
+
+        def raise_if_too_many_retries(stack_name, retries)
+          return if retries < 5
+          raise CloudFormationError, "Error waiting on stack operation for #{stack_name}"
+        end
+
         def update_stack_params(template)
           common_stack_params(template).merge(
             use_previous_template: false,

data/lib/aws_cft_tools/runbook.rb

@@ -48,6 +48,7 @@ module AwsCftTools
     def initialize(configuration = {})
       @options = configuration
       @client = AwsCftTools::Client.new(options)
+      @stdout = $stdout
     end
 
     # @!group Callbacks
@@ -137,6 +138,17 @@ module AwsCftTools
       end
     end
 
+    ##
+    # @param note [String] a debug note
+    #
+    # Prints the given content to stdout if running in +debug+ mode. Debug statements are output
+    # without any capture when running multiple threads.
+    #
+    def debug(note = nil)
+      return unless note && options[:debug]
+      @stdout.puts "DEBUG\nDEBUG  " + note.split(/\n/).join("\nDEBUG  ") + "\nDEBUG"
+    end
+
     ##
     # @param description [String] an optional verbose description
     # @yield runs the block if in +verbose+ mode

data/lib/aws_cft_tools/runbooks/common/changesets.rb

@@ -19,6 +19,7 @@ module AwsCftTools
         # provide a tabular report of changeset actions
         #
         def narrate_changes(changes)
+          TablePrint::Config.io = $stdout
           tp(
             changes.map(&:to_narrative),
             %i[action logical_id physical_id type replacement scopes]

data/lib/aws_cft_tools/runbooks/deploy.rb

@@ -48,19 +48,36 @@ module AwsCftTools
       end
 
       def process_slice(templates)
-        old_stdout = $stdout
-        threads = create_threads(templates) { |template| process_template(template) }
-        threads.map(&:thread).map(&:join)
-        puts threads.map(&:output).map(&:string)
+        jobs = options[:jobs]
+        if jobs && jobs > 1
+          process_slice_threaded(templates)
+        else
+          templates.each(&method(:process_template))
+        end
+      end
+
+      def process_slice_threaded(templates)
+        original_stdout = $stdout
+        $stdout = ThreadedOutput.new(original_stdout)
+        template_list = templates.map(&:name).join(', ')
+        debug("Creating threads for #{template_list}")
+        threads = create_threads(templates, &method(:process_template))
+        debug("Waiting on threads for #{template_list}")
+        threads.map(&:join)
       ensure
-        $stdout = old_stdout # just in case!
+        $stdout = original_stdout
       end
 
       def process_template(template)
+        ThreadedOutput.prefix = template_name = template.name
+        debug("Processing #{template_name}")
         is_update = deployed_templates.include?(template)
-        operation("#{is_update ? 'Updating' : 'Creating'}: #{template.name}") do
+        operation("#{is_update ? 'Updating' : 'Creating'}: #{template_name}") do
           exec_template(template: template, type: is_update ? :update : :create)
         end
+      ensure
+        $stdout.flush
+        debug("Finished processing #{template_name}")
       end
 
       def exec_template(params) # template:, type:

data/lib/aws_cft_tools/runbooks/deploy/threading.rb

@@ -1,5 +1,7 @@
 # frozen_string_literal: true
 
+require 'thread'
+
 module AwsCftTools
   module Runbooks
     class Deploy
@@ -9,27 +11,8 @@ module AwsCftTools
       module Threading
         private
 
-        # FIXME: things don't always work out well when capturing output
-        #        for now, we don't, and output gets mangled a bit when running with
-        #        multiple jobs in parallel
-        def with_captured_stdout(capture)
-          old_stdout = $stdout
-          old_table_io = TablePrint::Config.io
-          TablePrint::Config.io = $stdout = capture
-          yield
-        ensure
-          $stdout = old_stdout
-          TablePrint::Config.io = old_table_io
-        end
-
         def create_threads(list, &_block)
-          list.map { |item| threaded_process { yield item } }
-        end
-
-        def threaded_process(&block)
-          output = StringIO.new
-          thread = Thread.new { with_captured_stdout(output, &block) }
-          OpenStruct.new(output: output, thread: thread)
+          list.map { |item| Thread.new { yield item } }
         end
       end
     end

data/lib/aws_cft_tools/threaded_output.rb

@@ -0,0 +1,103 @@
+# frozen_string_literal: true
+
+require 'forwardable'
+require 'thread'
+
+module AwsCftTools
+  ##
+  # Provides a way to process output and prefix with a thread identifier. The object is shared by
+  # threads. Each thread should set its own prefix.
+  #
+  class ThreadedOutput
+    extend Forwardable
+
+    #
+    # @param real_stdout [IO] The file object that should be written to with prefixed text.
+    #
+    def initialize(real_stdout)
+      @stdout = real_stdout
+      @buffer = Hash.new { |hash, key| hash[key] = '' }
+      @semaphore = Mutex.new
+    end
+
+    def_delegator :@semaphore, :synchronize, :guarded
+    def_delegators ThreadedOutput, :prefix
+
+    ##
+    # The prefix for output from the current thread.
+    #
+    def self.prefix
+      Thread.current['output_prefix'] || ''
+    end
+
+    ##
+    # @param prefix [String] The prefix for each line of text output by the calling thread.
+    #
+    def self.prefix=(prefix)
+      Thread.current['output_prefix'] = prefix + ': '
+    end
+
+    ##
+    # Ensure all buffered text is output. If any text is output, a newline is output as well.
+    #
+    def flush
+      guarded { @stdout.puts prefix + buffer } if buffer != ''
+      self.buffer = ''
+    end
+
+    ##
+    # Write the string to the output with prefixes as appropriate. If the string does not end in a
+    # newline, then the remaining text will be buffered until a newline is seen.
+    #
+    def write(string)
+      print(string)
+    end
+
+    ##
+    # Writes all of the arguments to the output with prefixes. Appends a newline to each argument.
+    #
+    # @param args [Array<String>]
+    #
+    def puts(*args)
+      print(args.join("\n") + "\n")
+    end
+
+    ##
+    # Writes all of the arugments to the output without newlines. Will output the prefix after each
+    # newline.
+    #
+    # @param args [Array<String>]
+    #
+    def print(*args)
+      append(args.join(''))
+      printable_lines.each do |line|
+        guarded { @stdout.puts prefix + line }
+      end
+    end
+
+    private
+
+    def printable_lines
+      lines = buffer.split(/\n/)
+      if buffer[-1..-1] == "\n"
+        self.buffer = ''
+      else
+        self.buffer = lines.last
+        lines = lines[0..-2]
+      end
+      lines
+    end
+
+    def buffer
+      @buffer[prefix]
+    end
+
+    def append(value)
+      @buffer[prefix] += value
+    end
+
+    def buffer=(string)
+      @buffer[prefix] = string
+    end
+  end
+end

data/lib/aws_cft_tools/version.rb

@@ -4,5 +4,5 @@ module AwsCftTools
   ##
   # Version of AwsCftTools.
   #
-  VERSION = '0.1.0'
+  VERSION = '0.1.1'
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: aws-cft-tools
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.1.1
 platform: ruby
 authors:
 - Small Business Administration
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2017-11-29 00:00:00.000000000 Z
+date: 2017-12-15 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: aws-sdk
@@ -235,6 +235,7 @@ files:
 - ".travis.yml"
 - ".yardopts"
 - BEST-PRACTICES.md
+- CHANGELOG.md
 - CONTRIBUTING.md
 - Gemfile
 - LICENSE
@@ -292,6 +293,7 @@ files:
 - lib/aws_cft_tools/template_set/closure.rb
 - lib/aws_cft_tools/template_set/dependencies.rb
 - lib/aws_cft_tools/template_set/each_slice_state.rb
+- lib/aws_cft_tools/threaded_output.rb
 - lib/aws_cft_tools/version.rb
 - rubycritic.reek
 homepage: https://github.com/USSBA/aws-cft-tools