humidifier 2.15.0 → 3.0.0

data/LICENSE

@@ -1,6 +1,6 @@
 The MIT License
 
-Copyright (c) 2016 Localytics http://www.localytics.com
+Copyright (c) 2016-2019 Localytics http://www.localytics.com
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

data/README.md

@@ -7,48 +7,70 @@ Humidifier allows you to build AWS CloudFormation (CFN) templates programmatical
 
 For the full docs, go to [https://localytics.github.io/humidifier/](http://localytics.github.io/humidifier/). For local development instructions, see the [Development](https://localytics.github.io/humidifier/#label-Development) section.
 
-This project does *not* follow semantic versioning, but instead is linked to AWS' CloudFormation resource specification version since `1.2.1`. If there are developmental changes unrelated to bumping the resource specification, then they are released as another number incremented on the end of the resource specification version. As a result, breaking changes are reserved for whenever AWS updates with minor version changes.
+This project follows semantic versioning linked to AWS' CloudFormation resource specification version since `1.2.1`.
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'humidifier'
+```
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install humidifier
 
 ## Getting started
 
-Stacks are represented by the `Humidifier::Stack` class. You can set any of the top-level JSON attributes through the initializer. Resources are represented by an exact mapping from `AWS` resource names to `Humidifier` resources names (e.g. `AWS::EC2::Instance` becomes `Humidifier::EC2::Instance`). Resources have accessors for each JSON attribute. Each attribute can also be set through the `initialize`, `update`, and `update_attribute` methods.
+Stacks are represented by the `Humidifier::Stack` class. You can set any of the top-level JSON attributes (such as `name` and `description`) through the initializer.
+
+Resources are represented by an exact mapping from `AWS` resource names to `Humidifier` resources names (e.g. `AWS::EC2::Instance` becomes `Humidifier::EC2::Instance`). Resources have accessors for each JSON attribute. Each attribute can also be set through the `initialize`, `update`, and `update_attribute` methods.
 
 ### Example usage
 
-```ruby
-stack = Humidifier::Stack.new(name: 'Example-Stack', aws_template_format_version: '2010-09-09')
+The below example will create a stack with two resources, a loader balancer and an auto scaling group. It then deploys the new stack and pauses execution until the stack is finished being created.
 
-load_balancer = Humidifier::ElasticLoadBalancing::LoadBalancer.new(
-  listeners: [{ load_balancer_port: 80, protocol: 'http', instance_port: 80, instance_protocol: 'http' }]
+```ruby
+stack = Humidifier::Stack.new(name: 'Example-Stack')
+
+stack.add(
+  'LoaderBalancer',
+  Humidifier::ElasticLoadBalancing::LoadBalancer.new(
+    scheme: 'internal',
+    listeners: [
+      {
+        load_balancer_port: 80,
+        protocol: 'http',
+        instance_port: 80,
+        instance_protocol: 'http'
+      }
+    ]
+  )
 )
-load_balancer.scheme = 'internal'
 
-auto_scaling_group = Humidifier::AutoScaling::AutoScalingGroup.new(min_size: '1', max_size: '20')
-auto_scaling_group.update(
-  availability_zones: ['us-east-1a'],
-  load_balancer_names: [Humidifier.ref('LoadBalancer')]
+stack.add(
+  'AutoScalingGroup',
+  Humidifier::AutoScaling::AutoScalingGroup.new(
+    min_size: '1',
+    max_size: '20',
+    availability_zones: ['us-east-1a'],
+    load_balancer_names: [Humidifier.ref('LoadBalancer')]
+  )
 )
 
-stack.add('LoadBalancer', load_balancer)
-stack.add('AutoScalingGroup', auto_scaling_group)
 stack.deploy_and_wait
 ```
 
 ### Interfacing with AWS
 
-Once stacks have the appropriate resources, you can query AWS to handle all stack CRUD operations. The operations themselves are intuitively named (i.e. `create`, `update`, `delete`). There are also convenience methods for validating a stack body (`valid?`), checking the existence of a stack (`exists?`), and creating or updating based on existence (`deploy`). The `create`, `update`, `delete`, and `deploy` methods all have `_and_wait` corollaries that will cause the main ruby thread to sleep until the operation is complete.
-
-#### SDK version
-
-Humidifier assumes you have an `aws-sdk` gem installed when you call these operations. It detects the version of the gem you have installed and uses the appropriate API depending on what is available. If Humidifier cannot find any way to use the AWS SDK, it will warn you on every API call and simply return false.
-
-You can also manually specify the version of the SDK that you want to use, in the case that you have both gems in your load path. In that case, you would specify it on the Humidifier configuration object:
+Once stacks have the appropriate resources, you can query AWS to handle all stack CRUD operations. The operations themselves are intuitively named (i.e. `#create`, `#update`, `#delete`). There are also convenience methods for validating a stack body (`#valid?`), checking the existence of a stack (`#exists?`), and creating or updating based on existence (`#deploy`).
 
-```ruby
-Humidifier.configure do |config|
-  config.sdk_version = 1
-end
-```
+There are additionally four functions on `Humidifier::Stack` that support waiting for execution in AWS to finish. They all have non-blocking corollaries, and are named after them. They are: `#create_and_wait`, `#update_and_wait`, `#delete_and_wait`, and `#deploy_and_wait`.
 
 #### CloudFormation functions
 
@@ -56,19 +78,19 @@ You can use CFN intrinsic functions and references using `Humidifier.fn.[name]`
 
 #### Change Sets
 
-Instead of immediately pushing your changes to CloudFormation, Humidifier also supports change sets. Change sets are a powerful feature that allow you to see the changes that will be made before you make them. To read more about change sets see the [announcement article](https://aws.amazon.com/blogs/aws/new-change-sets-for-aws-cloudformation/). To use them in Humidifier, `Stack` has the `create_change_set` and `deploy_change_set` methods. The `create_change_set` method will create a change set on the stack. The `deploy_change_set` method will create a change set if the stack currently exists, and otherwise will create the stack.
+Instead of immediately pushing your changes to CloudFormation, Humidifier also supports change sets. Change sets are a powerful feature that allow you to see the changes that will be made before you make them. To read more about change sets see the [announcement article](https://aws.amazon.com/blogs/aws/new-change-sets-for-aws-cloudformation/). To use them in Humidifier, `Humidifier::Stack` has the `#create_change_set` and `#deploy_change_set` methods. The `#create_change_set` method will create a change set on the stack. The `#deploy_change_set` method will create a change set if the stack currently exists, and otherwise will create the stack.
 
 ### Introspection
 
-To see the template body, you can check the `to_cf` method on stacks, resources, fns, and refs. All of them will output a hash of what will be uploaded (except the stack, which will output a string representation).
+To see the template body, you can check the `#to_cf` method on stacks, resources, fns, and refs. All of them will output a hash of what will be uploaded (except the stack, which will output a string representation).
 
-Humidifier itself contains a registry of all possible resources that it supports. You can access it with `Humidifier.registry` which is a hash of AWS resource name pointing to the class.
+Humidifier itself contains a registry of all possible resources that it supports. You can access it with `Humidifier::registry` which is a hash of AWS resource name pointing to the class.
 
-Resources have an `aws_name` method to see how AWS references them. They also contain a `props` method that contains a hash of the name that Humidifier uses to reference the prop pointing to the appropriate prop object.
+Resources have an `::aws_name` method to see how AWS references them. They also contain a `::props` method that contains a hash of the name that Humidifier uses to reference the prop pointing to the appropriate prop object.
 
 ### Large templates
 
-When templates are especially large (larger than 51,200 bytes), they cannot be uploaded directly through the AWS SDK. You can configure Humidifier to seamlessly upload the templates to S3 and reference them using an S3 URL instead by:
+When templates are especially large (larger than 51,200 bytes), they cannot be uploaded directly through the AWS SDK. You can configure `Humidifier` to seamlessly upload the templates to S3 and reference them using an S3 URL instead by:
 
 ```ruby
 Humidifier.configure do |config|
@@ -80,13 +102,13 @@ end
 ### Forcing uploading
 
 You can force a stack to upload its template to S3 regardless of the size of the template. This is a useful option if you're going to be deploying multiple
-copies of a template or you just generally want a backup. You can set this option on a per-stack basis:
+copies of a template or if you want a backup. You can set this option on a per-stack basis:
 
 ```ruby
 stack.deploy(force_upload: true)
 ```
 
-or globally, but setting the configuration option:
+or globally, by setting the configuration option:
 
 ```ruby
 Humidifier.configure do |config|
@@ -96,7 +118,7 @@ end
 
 ## Development
 
-To get started, ensure you have ruby installed, version 2.1 or later. From there, install the `bundler` gem: `gem install bundler` and then `bundle install` in the root of the repository.
+To get started, ensure you have ruby installed, version 2.4 or later. From there, install the `bundler` gem: `gem install bundler` and then `bundle install` in the root of the repository.
 
 ### Testing
 

data/lib/humidifier/condition.rb

@@ -1,7 +1,6 @@
 # frozen_string_literal: true
 
 module Humidifier
-  # Represents a CFN stack condition
   class Condition
     attr_reader :opts
 
@@ -9,7 +8,6 @@ module Humidifier
       @opts = opts
     end
 
-    # CFN stack syntax
     def to_cf
       Serializer.dump(opts)
     end

data/lib/humidifier/config.rb

@@ -0,0 +1,46 @@
+# frozen_string_literal: true
+
+module Humidifier
+  # A container for user params
+  class Config
+    # If true, always upload the CloudFormation template to the configured S3
+    # destination. A useful option if you're going to be deploying multiple
+    # copies of a template or you just generally want a backup.
+    attr_accessor :force_upload
+
+    # The S3 bucket to which to deploy CloudFormation templates when
+    # `always_upload` is set to true or the template is too big for a string
+    # literal.
+    attr_accessor :s3_bucket
+
+    # An optional prefix for the JSON file names.
+    attr_accessor :s3_prefix
+
+    def initialize(opts = {})
+      @force_upload = opts[:force_upload]
+      @s3_bucket    = opts[:s3_bucket]
+      @s3_prefix    = opts[:s3_prefix]
+    end
+
+    # raise an error unless the s3_bucket field is set
+    # rubocop:disable Metrics/MethodLength
+    def ensure_upload_configured!(identifier)
+      return if s3_bucket
+
+      upload_message = <<~MSG
+        The %<identifier>s stack's body is too large to be use the template_body
+        option, and therefore must use the template_url option instead. You can
+        configure Humidifier to do this automatically by setting up the s3
+        config on the top-level Humidifier object like so:
+
+            Humidifier.configure do |config|
+              config.s3_bucket = 'my.s3.bucket'
+              config.s3_prefix = 'my-prefix/' # optional
+            end
+      MSG
+
+      raise upload_message.gsub('%<identifier>s', identifier)
+    end
+    # rubocop:enable Metrics/MethodLength
+  end
+end

data/lib/humidifier/fn.rb

@@ -1,13 +1,15 @@
 # frozen_string_literal: true
 
 module Humidifier
-  # Builds CFN function calls
   class Fn
     # The list of all internal functions provided by AWS from
     # http://docs.aws.amazon.com
     #   /AWSCloudFormation/latest/UserGuide/intrinsic-function-reference.html
-    FUNCTIONS = Utils.underscored(%w[And Base64 Equals FindInMap GetAtt GetAZs
-                                     If ImportValue Join Not Or Select Sub])
+    FUNCTIONS =
+      Humidifier.underscore(
+        %w[And Base64 Cidr Equals FindInMap GetAtt GetAZs If ImportValue Join
+           Not Or Select Split Sub Transform]
+      )
 
     attr_reader :name, :value
 
@@ -16,15 +18,12 @@ module Humidifier
       @value = value
     end
 
-    # CFN stack syntax
     def to_cf
       { name => Serializer.dump(value) }
     end
 
-    class << self
-      FUNCTIONS.each do |name, func|
-        define_method(func) { |value| new(name, value) }
-      end
+    FUNCTIONS.each do |name, func|
+      define_singleton_method(func) { |value| new(name, value) }
     end
   end
 end

data/lib/humidifier/loader.rb

@@ -1,25 +1,12 @@
 # frozen_string_literal: true
 
 module Humidifier
-  # Pre-setting this module because AWS has a "Config" module and the below
-  # register method dynamically looks up the module to see whether or not it
-  # exists, which before ruby 2.2 would result in the warning:
-  #   `const_defined?': Use RbConfig instead of obsolete and deprecated Config.
-  # @aws AWS::Config
-  module Config
-  end
-
   # Reads the specs/CloudFormationResourceSpecification.json file and load each
   # resource as a class
-  class Loader
-    filename = 'CloudFormationResourceSpecification.json'
-
-    # The path to the specification file
-    SPECPATH = File.expand_path(File.join('..', '..', '..', filename), __FILE__)
-
+  module Loader
     # Handles searching the PropertyTypes specifications for a specific
     # resource type
-    class StructureContainer
+    class PropertyTypes
       attr_reader :structs
 
       def initialize(structs)
@@ -40,46 +27,51 @@ module Humidifier
       end
     end
 
-    # loop through the specs and register each class
-    def load
-      parsed = JSON.parse(File.read(SPECPATH))
-      structs = StructureContainer.new(parsed['PropertyTypes'])
+    class << self
+      # loop through the specs and register each class
+      def load
+        parsed = parse_spec
+        types = PropertyTypes.new(parsed['PropertyTypes'])
 
-      parsed['ResourceTypes'].each do |key, spec|
-        match = key.match(/\A(\w+)::(\w+)::(\w+)\z/)
+        parsed['ResourceTypes'].each do |key, spec|
+          match = key.match(/\A(\w+)::(\w+)::(\w+)\z/)
+          register(match[1], match[2], match[3], spec, types.search(key))
+        end
 
-        register(match[1], match[2], match[3], spec, structs.search(key))
+        Humidifier.registry.freeze
       end
-    end
-
-    # convenience class method to build a new loader and call load
-    def self.load
-      new.load
-    end
 
-    private
+      private
 
-    def build_class(aws_name, spec, substructs)
-      Class.new(Resource) do
-        self.aws_name = aws_name
-        self.props =
-          spec['Properties'].map do |(key, config)|
-            prop = Props.from(key, config, substructs)
-            [prop.name, prop]
-          end.to_h
+      def build_class(aws_name, spec, substructs)
+        Class.new(Resource) do
+          self.aws_name = aws_name
+          self.props =
+            spec['Properties'].map do |(key, config)|
+              prop = Props.from(key, config, substructs)
+              [prop.name, prop]
+            end.to_h
+        end
       end
-    end
 
-    def register(top, group, resource, spec, substructs)
-      aws_name = "#{top}::#{group}::#{resource}"
-      resource_class = build_class(aws_name, spec, substructs)
+      def parse_spec
+        relative =
+          File.join('..', '..', 'CloudFormationResourceSpecification.json')
 
-      unless Humidifier.const_defined?(group)
-        Humidifier.const_set(group, Module.new)
+        JSON.parse(File.read(File.expand_path(relative, __dir__)))
       end
 
-      Humidifier.const_get(group).const_set(resource, resource_class)
-      Humidifier.registry[aws_name] = resource_class
+      def register(top, group, resource, spec, substructs)
+        aws_name = "#{top}::#{group}::#{resource}"
+        resource_class = build_class(aws_name, spec, substructs)
+
+        unless Humidifier.const_defined?(group)
+          Humidifier.const_set(group, Module.new)
+        end
+
+        Humidifier.const_get(group).const_set(resource, resource_class)
+        Humidifier.registry[aws_name] = resource_class
+      end
     end
   end
 end

data/lib/humidifier/mapping.rb

@@ -1,7 +1,6 @@
 # frozen_string_literal: true
 
 module Humidifier
-  # Represents a CFN stack mapping
   class Mapping
     attr_reader :opts
 
@@ -9,7 +8,6 @@ module Humidifier
       @opts = opts
     end
 
-    # CFN stack syntax
     def to_cf
       Serializer.dump(opts)
     end

data/lib/humidifier/output.rb

@@ -1,7 +1,6 @@
 # frozen_string_literal: true
 
 module Humidifier
-  # Represents a CFN stack output
   class Output
     attr_reader :description, :value, :export_name
 
@@ -11,12 +10,11 @@ module Humidifier
       @export_name = opts[:export_name]
     end
 
-    # CFN stack syntax
     def to_cf
-      cf = { 'Value' => Serializer.dump(value) }
-      cf['Description'] = description if description
-      cf['Export'] = { 'Name' => export_name } if export_name
-      cf
+      { 'Value' => Serializer.dump(value) }.tap do |cf|
+        cf['Description'] = description if description
+        cf['Export'] = { 'Name' => export_name } if export_name
+      end
     end
   end
 end

data/lib/humidifier/parameter.rb

@@ -1,13 +1,12 @@
 # frozen_string_literal: true
 
 module Humidifier
-  # Represents a CFN stack parameter
   class Parameter
-    # The allowed properties of all stack parameters
     PROPERTIES =
-      Utils.underscored(%w[AllowedPattern AllowedValues ConstraintDescription
-                           Default Description MaxLength MaxValue MinLength
-                           MinValue NoEcho])
+      Humidifier.underscore(
+        %w[AllowedPattern AllowedValues ConstraintDescription Default
+           Description MaxLength MaxValue MinLength MinValue NoEcho]
+      )
 
     attr_reader :type, *PROPERTIES.values
 
@@ -21,12 +20,12 @@ module Humidifier
 
     # CFN stack syntax
     def to_cf
-      cf = { 'Type' => type }
-      PROPERTIES.each do |name, prop|
-        val = send(prop)
-        cf[name] = Serializer.dump(val) if val
+      { 'Type' => type }.tap do |cf|
+        PROPERTIES.each do |name, prop|
+          value = public_send(prop)
+          cf[name] = Serializer.dump(value) if value
+        end
       end
-      cf
     end
   end
 end

data/lib/humidifier/props.rb

@@ -1,8 +1,183 @@
 # frozen_string_literal: true
 
 module Humidifier
-  # Container for property of CFN resources
   module Props
+    # Superclass for all CFN properties
+    class Prop
+      # The list of classes that are valid beyond the normal values for each
+      # prop
+      WHITELIST = [Fn, Ref].freeze
+
+      attr_reader :key, :name, :spec
+
+      def initialize(key, spec = {})
+        @key  = key
+        @name = key.underscore
+        @spec = spec
+      end
+
+      # the link to the AWS docs
+      def documentation
+        spec['Documentation']
+      end
+
+      # true if this property is required by the resource
+      def required?
+        spec['Required']
+      end
+
+      # CFN stack syntax
+      def to_cf(value)
+        [key, Serializer.dump(value)]
+      end
+
+      # the type of update that occurs when this property is updated on its
+      # associated resource
+      def update_type
+        spec['UpdateType']
+      end
+
+      def valid?(value)
+        self.class.allowed_types.any? { |type| value.is_a?(type) }
+      end
+
+      class << self
+        def allowed_types
+          @allowed_types ||= [Fn, Ref]
+        end
+
+        def allow_type(*types)
+          allowed_types
+          @allowed_types += types
+        end
+      end
+    end
+
+    class BooleanProp < Prop
+      allow_type TrueClass, FalseClass
+    end
+
+    class DoubleProp < Prop
+      allow_type Integer, Float
+    end
+
+    class IntegerProp < Prop
+      allow_type Integer
+    end
+
+    class JsonProp < Prop
+      allow_type Hash
+    end
+
+    class StringProp < Prop
+      allow_type String
+    end
+
+    class TimestampProp < Prop
+      allow_type Time, Date
+    end
+
+    class ListProp < Prop
+      attr_reader :subprop
+
+      def initialize(key, spec = {}, substructs = {})
+        super(key, spec)
+        @subprop = Props.singular_from(key, spec, substructs)
+      end
+
+      def to_cf(list)
+        cf_value =
+          if list.respond_to?(:to_cf)
+            list.to_cf
+          else
+            list.map { |value| subprop.to_cf(value).last }
+          end
+
+        [key, cf_value]
+      end
+
+      def valid?(list)
+        return true if super(list)
+
+        list.is_a?(Enumerable) && list.all? { |value| subprop.valid?(value) }
+      end
+    end
+
+    class MapProp < Prop
+      attr_reader :subprop
+
+      def initialize(key, spec = {}, substructs = {})
+        super(key, spec)
+        @subprop = Props.singular_from(key, spec, substructs)
+      end
+
+      def to_cf(map)
+        cf_value =
+          if map.respond_to?(:to_cf)
+            map.to_cf
+          else
+            map.map do |subkey, subvalue|
+              [subkey, subprop.to_cf(subvalue).last]
+            end.to_h
+          end
+
+        [key, cf_value]
+      end
+
+      def valid?(map)
+        return true if super(map)
+
+        map.is_a?(Hash) && map.values.all? { |value| subprop.valid?(value) }
+      end
+    end
+
+    class StructureProp < Prop
+      attr_reader :subprops
+
+      def initialize(key, spec = {}, substructs = {})
+        super(key, spec)
+        @subprops = subprops_from(substructs, spec['ItemType'] || spec['Type'])
+      end
+
+      def to_cf(struct)
+        cf_value =
+          if struct.respond_to?(:to_cf)
+            struct.to_cf
+          else
+            struct.map do |subkey, subvalue|
+              subprops[subkey.to_s].to_cf(subvalue)
+            end.to_h
+          end
+
+        [key, cf_value]
+      end
+
+      def valid?(struct)
+        super(struct) || (struct.is_a?(Hash) && valid_struct?(struct))
+      end
+
+      private
+
+      def subprops_from(substructs, type)
+        subprop_names = substructs.fetch(type, {}).fetch('Properties', {})
+
+        subprop_names.each_with_object({}) do |(key, config), subprops|
+          subprops[key.underscore] =
+            if config['ItemType'] == type
+              self
+            else
+              Props.from(key, config, substructs)
+            end
+        end
+      end
+
+      def valid_struct?(struct)
+        struct.all? do |key, value|
+          subprops.key?(key.to_s) && subprops[key.to_s].valid?(value)
+        end
+      end
+    end
+
     class << self
       # builds the appropriate prop object from the given spec line
       def from(key, spec, substructs = {})
@@ -19,7 +194,7 @@ module Humidifier
       def singular_from(key, spec, substructs)
         primitive = spec['PrimitiveItemType'] || spec['PrimitiveType']
 
-        if primitive
+        if primitive && !%w[List Map].include?(primitive)
           primitive = 'Integer' if primitive == 'Long'
           const_get(:"#{primitive}Prop").new(key, spec)
         else