humidifier 3.0.1 → 3.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 4981fed9d0ee50e48fb96d62dc82574b5f66c4a3c8f96eb45a7a36babde88900
-  data.tar.gz: 1106c9c7ed5b0ed6a6b3252bd6377d0b73bdaf848c23a9fa0115f03ceb79392e
+  metadata.gz: 70a3a43a433113df153ba4a97f549ac007299f91fb5c0a266dea0cbb70303712
+  data.tar.gz: 1e81ad0413783a00d1c0067334319634ceac86185de55e73fbe2c747f237b328
 SHA512:
-  metadata.gz: 9b25d56506a612c4a1962def35bc044b09977258eb5ce64f4a998636b8877930d00c7e0d18151af072be8a2a1656e9883a7d6a809d235df9c36d637622fcba25
-  data.tar.gz: 555aec2670276edba2614dd9776a4eb7e265a772bb2a58a98beb1afc3ac301b266d922235b4d61339a84aa7bc7e68db0ba36cb95fd5524750fafa59dcc88ea82
+  metadata.gz: 7626be05d2483a66ab1909fcf7bb515609fc5eb7fb404b648b10554af1501fb801c0625250258d2675ed056803acfc4a42d238f8ca1696447c1dc16165da795d
+  data.tar.gz: 4f9d3167302db844968d433b12e82c8128f538fad1f27da6d0ecdcd8cd7398c5d8fd2fc36da15bf03bd1a2334b065307a4709e7f7e10aeb22e1198345da8d56f

data/README.md

@@ -5,7 +5,35 @@
 
 Humidifier allows you to build AWS CloudFormation (CFN) templates programmatically. CFN stacks and resources are represented as Ruby objects with accessors for all their supported properties. Stacks and resources have `to_cf` methods that allow you to quickly inspect what will be uploaded.
 
-For the full docs, go to [https://localytics.github.io/humidifier/](http://localytics.github.io/humidifier/). For local development instructions, see the [Development](https://localytics.github.io/humidifier/#label-Development) section.
+- [Installation](#installation)
+- [Getting started](#getting-started)
+  - [Example usage](#example-usage)
+  - [Interfacing with AWS](#interfacing-with-aws)
+    - [CloudFormation functions](#cloudformation-functions)
+    - [Change Sets](#change-sets)
+  - [Introspection](#introspection)
+  - [Large templates](#large-templates)
+  - [Forcing uploading](#forcing-uploading)
+- [CLI](#cli)
+  - [Resource files](#resource-files)
+  - [Mappers](#mappers)
+  - [Using the CLI](#using-the-cli)
+    - [`change [?stack]`](#change-stack)
+    - [`deploy [?stack] [*parameters]`](#deploy-stack-parameters)
+    - [`display [stack] [?pattern]`](#display-stack-pattern)
+    - [`stacks`](#stacks)
+    - [`upload [?stack]`](#upload-stack)
+    - [`validate [?stack]`](#validate-stack)
+  - [Parameters](#parameters)
+  - [Shortcuts](#shortcuts)
+    - [Automatic id properties](#automatic-id-properties)
+    - [Anonymous mappers](#anonymous-mappers)
+    - [Cross-stack references](#cross-stack-references)
+- [Development](#development)
+  - [Testing](#testing)
+  - [Specs](#specs)
+  - [Contributing](#contributing)
+  - [License](#license)
 
 ## Installation
 
@@ -72,7 +100,7 @@ There are additionally four functions on `Humidifier::Stack` that support waitin
 
 #### CloudFormation functions
 
-You can use CFN intrinsic functions and references using `Humidifier.fn.[name]` and `Humidifier.ref`. Those will build appropriate structures that know how to be dumped to CFN syntax appropriately.
+You can use CFN intrinsic functions and references using `Humidifier.fn.[name]` and `Humidifier.ref`. They will build appropriate structures that know how to be dumped to CFN syntax.
 
 #### Change Sets
 
@@ -114,6 +142,241 @@ Humidifier.configure do |config|
 end
 ```
 
+## CLI
+
+`Humidifier` can also be used as a CLI for managing resources through configuration files. To get started, build a ruby script (for example `bin/humidifier`) that executes the `Humidifier::CLI` class, like so:
+
+```ruby
+#!/usr/bin/env ruby
+require 'humidifier'
+
+Humidifier.configure do |config|
+  # optional, defaults to the current working directory, so that all of the
+  # directories from the location that you run the CLI are assumed to contain
+  # resource specifications
+  config.stack_path = 'stacks'
+
+  # optional, a default prefix to use before deploying to AWS
+  config.stack_prefix = 'humidifier-'
+
+  # specifies that `users.yml` files contain specifications for `AWS::IAM::User`
+  # resources
+  config.map :users, to: 'IAM::User'
+end
+
+Humidifier::CLI.start(ARGV)
+```
+
+### Resource files
+
+Inside of the `stacks` directory configured above, create a subdirectory for each CloudFormation stack that you want to deploy. With the above configuration, we can create YAML files in the form of `users.yml` for each stack, which will specify IAM users to create. The file format looks like the below:
+
+```yaml
+EngUser:
+  path: /humidifier/
+  user_name: EngUser
+  groups:
+  - Engineering
+  - Testing
+  - Deployment
+
+AdminUser:
+  path: /humidifier/
+  user_name: AdminUser
+  groups:
+  - Management
+  - Administration
+```
+
+The top-level keys are the logical resource names that will be displayed in the CloudFormation screen. They point to a map of key/value pairs that will be passed on to `humidifier`. Any `humidifier` (and therefore any CloudFormation) attribute may be specified. For more information on CloudFormation templates and which attributes may be specified, see both the [`humidifier` docs](http://localytics.github.io/humidifier) and the [CloudFormation docs](http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-guide.html).
+
+### Mappers
+
+Oftentimes, specifying these attributes can become repetitive, e.g., each user should automatically receive the same "path" attribute. Other times, you may want custom logic to execute depending on which AWS environment you're running in. Finally, you may want to reference resources in the same or other stacks.
+
+`Humidifier`'s solution for this is to allow customized "mapper" classes to take the user-provided attributes and transform them into the attributes that CloudFormation expects. Consider the following example for mapping a user:
+
+```ruby
+class UserMapper < Humidifier::Config::Mapper
+  GROUPS = {
+    'eng' => %w[Engineering Testing Deployment],
+    'admin' => %w[Management Administration]
+  }
+
+  defaults do |logical_name|
+    { path: '/humidifier/', user_name: logical_name }
+  end
+
+  attribute :group do |group|
+    groups = GROUPS[group]
+    groups.any? ? { groups: GROUPS[group] } : {}
+  end
+end
+
+Humidifier.configure do |config|
+  config.map :users, to: 'IAM::User', using: UserMapper
+end
+```
+
+This means that by default, all entries in the `users.yml` files will get a `/humidifier/` path, the `user_name` attribute will be set based on the logical name that was provided for the resource, and you can additionally specify a `group` attribute, even though it is not native to CloudFormation. With this `group` attribute, it will actually map to the `groups` attribute that CloudFormation expects.
+
+With this new mapper in place, we can simplify our YAML file to:
+
+```yaml
+EngUser:
+  group: eng
+
+AdminUser:
+  group: admin
+```
+
+### Using the CLI
+
+Now that you've configured your CLI, your resources, and your mappers, you can use the CLI to display, validate, and deploy your infrastructure to CloudFormation. Run your script without any arguments to get the help message and explanations for each command.
+
+Each command has an `--aws-profile` (or `-p`) option for specifying which profile to authenticate against when querying AWS. You should ensure that this profile has the correct permissions for creating whatever resources are going to part of your stack. You can also rely on the `AWS_*` environment variables, or the EC2 instance profile if you're deploying from an instance. For more information, see the [AWS docs](http://docs.aws.amazon.com/sdkforruby/api/) under the "Configuration" section.
+
+Below are the list of commands and some of their options.
+
+#### `change [?stack]`
+
+Creates a change set for either the specified stack or all stacks in the repo. The change set represents the changes between what is currently deployed versus 
+the resources represented by the configuration.
+
+#### `deploy [?stack] [*parameters]`
+
+Creates or updates (depending on if the stack already exists) one or all stacks in the repo.
+
+The `deploy` command also allows a `--prefix` command line argument that will override the default prefix (if one is configured) for the stack that is being deployed. This is especially useful when you're deploying multiple copies of the same stack (for instance, multiple autoscaling groups) that have different purposes or semantically mean newer versions of resources.
+
+#### `display [stack] [?pattern]`
+
+Displays the specified stack in JSON format on the command line. If you optionally pass a pattern argument, it will filter the resources down to just ones whose names match the given pattern.
+
+#### `stacks`
+
+Displays the names of all of the stacks that `humidifier` is managing.
+
+#### `upload [?stack]`
+
+Upload one or all stacks in the repo to S3 for reference later. Note that this must be combined with the `humidifier` `s3_bucket` configuration option.
+
+#### `validate [?stack]`
+
+Validate that one or all stacks in the repo are properly configured and using values that CloudFormation understands.
+
+### Parameters
+
+CloudFormation template parameters can be specified by having a special `parameters.yml` file in your stack directory. This file should contain a YAML-encoded object whose keys are the names of the parameters and whose values are the parameter configuration (using the same underscore paradigm as `humidifier` resources for specifying configuration).
+
+You can pass values to the CLI deploy command after the stack name on the command line as in:
+
+```sh
+bin/humidifier deploy foobar Param1=Foo Param2=Bar
+```
+
+Those parameters will get passed in as values when the stack is deployed.
+
+### Shortcuts
+
+A couple of convenient shortcuts are built into `humidifier` so that writing templates and mappers both can be more concise.
+
+#### Automatic id properties
+
+There are a lot of properties in the AWS CloudFormation resource specification that are simply pointers to other entities within the AWS ecosystem. For example, an `AWS::EC2::VPCGatewayAttachment` entity has a `VpcId` property that represents the ID of the associated `AWS::EC2::VPC`.
+
+Because this pattern is so common, `humidifier` detects all properties ending in `Id` and allows you to specify them without the suffix. If you choose to use this format, `humidifier` will automatically turn that value into a [CloudFormation resource reference](http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-ref.html).
+
+#### Anonymous mappers
+
+A lot of the time, mappers that you create will not be overly complicated, especially if you're using automatic id properties. So, the `config.map` method optionally takes a block, and allows you to specify the mapper inline. This is recommended for mappers that aren't too complicated as to warrant their own class (for instance, for testing purposes). An example of this using the `UserMapper` from above is below:
+
+```ruby
+Humidifier.configure do |config|
+  config.map :users, to: 'IAM::User' do
+    GROUPS = {
+      'eng' => %w[Engineering Testing Deployment],
+      'admin' => %w[Management Administration]
+    }
+
+    defaults do |logical_name|
+      { path: '/humidifier/', user_name: logical_name }
+    end
+
+    attribute :group do |group|
+      groups = GROUPS[group]
+      groups.any? ? { groups: GROUPS[group] } : {}
+    end
+  end
+end
+```
+
+#### Cross-stack references
+
+AWS allows cross-stack references through the [intrinsic `Fn::ImportValue` function](http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-importvalue.html). You can take advantage of this with `humidifier` by using the `export: true` option on resources in your stacks. For instance, if in one stack you have a subnet that you need to reference in another, you could (`stacks/vpc/subnets.yml`):
+
+```yaml
+ProductionPrivateSubnet2a:
+  vpc: ProductionVPC
+  cidr_block: 10.0.0.0/19
+  availability_zone: us-west-2a
+  export: true
+
+ProductionPrivateSubnet2b:
+  vpc: ProductionVPC
+  cidr_block: 10.0.64.0/19
+  availability_zone: us-west-2b
+  export: true
+
+ProductionPrivateSubnet2c:
+  vpc: ProductionVPC
+  cidr_block: 10.0.128.0/19
+  availability_zone: us-west-2c
+  export: true
+```
+
+And then in another stack, you could reference those values (`stacks/rds/db_subnets_groups.yml`):
+
+```yaml
+ProductionDBSubnetGroup:
+  db_subnet_group_description: Production DB private subnet group
+  subnets:
+  - ProductionPrivateSubnet2a
+  - ProductionPrivateSubnet2b
+  - ProductionPrivateSubnet2c
+```
+
+Within the configuration, you would specify to use the `Fn::ImportValue` function like so:
+
+```ruby
+Humidifier.configure do |config|
+  config.stack_path = 'stacks'
+
+  config.map :subnets, to: 'EC2::Subnet'
+
+  config.map :db_subnet_groups, to: 'RDS::DBSubnetGroup' do
+    attribute :subnets do |subnet_names|
+      subnet_ids =
+        subnet_names.map do |subnet_name|
+          Humidifier.fn.import_value(subnet_name)
+        end
+
+      { subnet_ids: subnet_ids }
+    end
+  end
+end
+```
+
+If you specify `export: true` it will by default export a reference to the resource listed in the stack. You can also choose to export a different attribute by specifying the attribute as the value to export. For example, if we were creating instance profiles and wanted to export the `Arn` so that it could be referenced by an instance later, we could:
+
+```yaml
+APIRoleInstanceProfile:
+  depends_on: APIRole
+  roles:
+  - APIRole
+  export: Arn
+```
+
 ## Development
 
 To get started, ensure you have ruby installed, version 2.4 or later. From there, install the `bundler` gem: `gem install bundler` and then `bundle install` in the root of the repository.

data/lib/humidifier/cli.rb

@@ -0,0 +1,107 @@
+# frozen_string_literal: true
+
+module Humidifier
+  # A CLI for running commands to manipulate the stacks that Humidifier knows
+  # about.
+  class CLI < Thor
+    class_option :aws_profile, desc: 'The AWS profile to authenticate with',
+                               aliases: ['-p']
+
+    class_option :debug, desc: 'Sets up debug mode', aliases: ['-d']
+    class_around :safe_execute
+
+    desc 'change [?stack]', 'Create changesets for one or all stacks'
+    def change(name = nil)
+      authorize
+
+      stack_names_from(name).each do |stack_name|
+        directory = Directory.new(stack_name)
+
+        puts "Creating a changeset for #{directory.stack_name}"
+        directory.create_change_set
+      end
+    end
+
+    desc 'deploy [?stack] [*parameters]', 'Update one or all stacks'
+    option :wait, desc: 'Wait for the stack to create/update',
+                  type: :boolean, default: false
+    option :prefix, desc: 'The prefix to use for the stack'
+    def deploy(name = nil, *parameters)
+      authorize
+
+      stack_names_from(name).each do |stack_name|
+        directory = Directory.new(stack_name, prefix: options[:prefix])
+
+        puts "Deploying #{directory.stack_name}"
+        directory.deploy(options[:wait], parameters_from(parameters))
+      end
+    end
+
+    desc 'display [stack] [?pattern]',
+         'Display the CloudFormation JSON for a given stack'
+    def display(name, pattern = nil)
+      puts Directory.new(name, pattern: pattern && /#{pattern}/i).to_cf
+    end
+
+    desc 'stacks', 'List the stacks known to Humidifier'
+    def stacks
+      puts Humidifier.config.stack_names.sort
+    end
+
+    desc 'upload [?stack]', 'Upload one or all stacks to S3'
+    def upload(name = nil)
+      authorize
+
+      stack_names_from(name).each do |stack_name|
+        directory = Directory.new(stack_name)
+
+        puts "Uploading #{directory.stack_name}"
+        directory.upload
+      end
+    end
+
+    desc 'validate [?stack]',
+         'Validate that one or all stacks are valid with CloudFormation'
+    def validate(name = nil)
+      authorize
+
+      print 'Validating... '
+
+      valid =
+        stack_names_from(name).all? do |stack_name|
+          Directory.new(stack_name).valid?
+        end
+
+      puts valid ? 'Valid.' : 'Invalid.'
+    end
+
+    no_commands do
+      def authorize
+        return unless options[:aws_profile]
+
+        Aws.config[:credentials] =
+          Aws::SharedCredentials.new(profile_name: options[:aws_profile])
+      end
+
+      def parameters_from(opts)
+        opts.map do |opt|
+          key, value = opt.split('=')
+          { parameter_key: key, parameter_value: value }
+        end
+      end
+
+      def safe_execute
+        yield
+      rescue Error => error
+        raise error if options[:debug]
+
+        puts error.message
+        exit 1
+      end
+
+      def stack_names_from(name)
+        name ? [name] : Humidifier.config.stack_names
+      end
+    end
+  end
+end

data/lib/humidifier/config/mapper.rb

@@ -0,0 +1,116 @@
+# frozen_string_literal: true
+
+module Humidifier
+  class Config
+    # The parent class for mapper classes. These classes are used to transform
+    # arbitrary attributes coming from the user-provided YAML files into valid
+    # CloudFormation props that can then be used in the template. This class
+    # provides an easy-to-extend DSL that allows for default attributes
+    # specifying custom attributes.
+    class Mapper
+      # Raised when the attribute given in the file could not be matched to an
+      # attribute in the mapper.
+      class InvalidResourceAttributeError < Error
+        def initialize(clazz, key)
+          super("Invalid attribute name given for #{clazz.aws_name}: #{key}")
+        end
+      end
+
+      # The list of attributes that are common to all resources that need to be
+      # handled separately.
+      COMMON_ATTRIBUTES = Resource::COMMON_ATTRIBUTES.values
+
+      class << self
+        # Defines a custom attribute. The given block will receive the
+        # user-provided value for the attribute. The block should return a hash
+        # where the keys are valid humidifier properties and the values are
+        # valid values for those properties. In the below example, we specify
+        # the group attribute which maps to the groups attribute after some
+        # transformation.
+        #
+        #     attribute :group do |group|
+        #       groups = GROUPS[group]
+        #       groups.any? ? { groups: GROUPS[group] } : {}
+        #     end
+        def attribute(name, &block)
+          define_method(:"attribute_#{name}", &block)
+          attribute_methods << name
+        end
+
+        # The names of the custom attribute methods.
+        def attribute_methods
+          @attribute_methods ||= []
+        end
+
+        # Defines the default attributes that should be applied to all resources
+        # of this type. The given block will be passed the logical resource
+        # name that the user specified for the resource. The block should return
+        # a hash where the keys are valid humidifier properties and the values
+        # are valid values for those properties. In the example below, the
+        # user_name property is set based on the logical name.
+        #
+        #     defaults do |name|
+        #       { user_name: name }
+        #     end
+        def defaults(&block)
+          define_method(:attribute_defaults, &block)
+        end
+      end
+
+      # Builds a humidifier resource using the given humidifier resource class,
+      # the logical name for the resource, and the user-specified attributes.
+      def resource_for(clazz, name, attributes)
+        mapped =
+          respond_to?(:attribute_defaults) ? attribute_defaults(name) : {}
+
+        attributes.each do |key, value|
+          mapped.merge!(mapped_from(clazz, key, value))
+        end
+
+        common_attributes = common_attributes_from(mapped)
+
+        resource = clazz.new(mapped)
+        resource.update_attributes(common_attributes)
+        resource
+      end
+
+      private
+
+      def common_attributes_from(mapped)
+        COMMON_ATTRIBUTES.each_with_object({}) do |common_attribute, extract|
+          extracted = mapped.delete(common_attribute)
+          extract[common_attribute] = extracted if extracted
+        end
+      end
+
+      def mapped_from(clazz, key, value) # rubocop:disable Metrics/MethodLength
+        if self.class.attribute_methods.include?(key.to_sym)
+          # The given attribute name has been defined using the `::attribute`
+          # DSL method, so send the given value to that method and return the
+          # resulting hash.
+          public_send(:"attribute_#{key}", value)
+        elsif clazz.prop?(key)
+          # The given attribute name is a valid property on the resource, so
+          # directly map the attribute to the given value.
+          { key.to_sym => value }
+        elsif clazz.prop?("#{key}_id")
+          # The given attribute name corresponds to a property on the resource
+          # that takes the ID of another resource (for example, specifying the
+          # vpc option in the file when the resource has a vpc_id property). In
+          # this case, automatically convert the given value into a
+          # CloudFormation reference.
+          { "#{key}_id": Humidifier.ref(value) }
+        elsif COMMON_ATTRIBUTES.include?(key.to_sym)
+          # The given attribute name is one of the attributes common to all
+          # resources (for example creation_policy), so map that directly to the
+          # given value.
+          { key.to_sym => value }
+        else
+          # The given attribute name did not match one of the valid options, so
+          # raise an error to alert the user.
+          raise InvalidResourceAttributeError.new(clazz, key)
+        end
+      end
+    end
+  end
+end

data/lib/humidifier/config/mapping.rb

@@ -0,0 +1,40 @@
+# frozen_string_literal: true
+
+module Humidifier
+  class Config
+    class Mapping
+      attr_reader :clazz, :mapper
+
+      def initialize(opts = {}, &block)
+        @clazz = Humidifier[normalized(opts[:to])]
+        raise Error, "Invalid resource: #{opts[:to].inspect}" if @clazz.nil?
+
+        if opts[:using] && block_given?
+          raise Error, 'Cannot specify :using and provide an anonymous mapper'
+        end
+
+        @mapper = mapper_from(opts, &block)
+      end
+
+      def resource_for(name, attributes)
+        mapper.resource_for(clazz, name, attributes)
+      end
+
+      private
+
+      def mapper_from(opts, &block)
+        if opts[:using]
+          opts[:using].new
+        elsif block_given?
+          Class.new(Mapper, &block).new
+        else
+          Mapper.new
+        end
+      end
+
+      def normalized(name)
+        name.start_with?('AWS') ? name : "AWS::#{name}"
+      end
+    end
+  end
+end

data/lib/humidifier/config.rb

@@ -16,31 +16,110 @@ module Humidifier
     # An optional prefix for the JSON file names.
     attr_accessor :s3_prefix
 
-    def initialize(opts = {})
-      @force_upload = opts[:force_upload]
-      @s3_bucket    = opts[:s3_bucket]
-      @s3_prefix    = opts[:s3_prefix]
+    # The path to the various directories containing the YAML files representing
+    # stacks. If blank, it's assumed to be the current working direction from
+    # which the CLI is executing.
+    attr_reader :stack_path
+
+    # An optional prefix for the stack names before they get uploaded to AWS.
+    attr_accessor :stack_prefix
+
+    def initialize
+      @mappings = {}
+      @stack_path = '.'
+    end
+
+    def files_for(name)
+      Dir["#{stack_path}/#{name}/*.yml"]
+    end
+
+    # `#map` is a declaration of a link between a file name and a mapper
+    # configuration. It is used to declare the manner in which a set of
+    # attributes read from a resource file is converted into instantiations of
+    # that type.
+    #
+    # For more information about the mapping DSL and how attributes get
+    # converted into props, see the `Humidifier::Config::Mapper` class.
+    #
+    # == Basic mapping
+    #
+    # For the most basic of mappings, you can just map a file name to a
+    # resource, which effectively means that each attribute you provide must
+    # map directly to an AWS attribute for that resource, and that no additional
+    # attributes will be provided. For example, the following code indicates
+    # that files named `routes.yml` will contain `AWS::EC2::Route` resources,
+    # and that every attribute read will directly correspond to one from AWS:
+    #
+    #     Humidifier.configure do |config|
+    #       config.map :routes, to: 'EC2::Route'
+    #     end
+    #
+    # == Using the DSL
+    #
+    # For mappings for which you want to use the `Humidifier::Config::Mapper`
+    # DSL, you can pass a block which will get used to create a new mapper
+    # class. This is useful for shorter mapper declarations. For example, in the
+    # following code we map files named `instance_profiles.yml` to
+    # `AWS::IAM::InstanceProfile` resources. In that mapping we default the
+    # `path` prop to "/" and we allow the `roles` prop to just pass a list of
+    # roles as an array, which we then convert into CloudFormation references.
+    #
+    #     Humidifier.configure do |config|
+    #       config.map :instance_profiles, to: 'IAM::InstanceProfile' do
+    #         defaults do |_|
+    #           { path: '/' }
+    #         end
+    #
+    #         attribute :roles do |names|
+    #           { roles: names.map { |name| Humidifier.ref(name) } }
+    #         end
+    #       end
+    #     end
+    #
+    # == Reusing mappers
+    #
+    # Finally, if you want to pull the mapper out for reuse, testing, or just
+    # separation of code, you can pass the `:using` key with a mapper as a
+    # value. This will cause the given file type to be mapped using whatever
+    # class you provided. For example, the following code creates a mapper that
+    # automatically tags the resource with the logical name from the stack. It
+    # then configures network ACLs to use that so that all network ACL resource
+    # declarations automatically have a tag on them with their name.
+    #
+    #     class NameToTag < Humidifier::Config::Mapper
+    #       defaults do |logical_name|
+    #         { tags: [{ key: 'Name', value: logical_name }] }
+    #       end
+    #     end
+    #
+    #     Humidifier.configure do |config|
+    #       config.map :network_acls, to: 'EC2::NetworkAcl', using: NameToTag
+    #     end
+    #
+    def map(type, opts = {}, &block)
+      mappings[type.to_sym] = Mapping.new(opts, &block)
     end
 
-    # raise an error unless the s3_bucket field is set
-    # rubocop:disable Metrics/MethodLength
-    def ensure_upload_configured!(identifier)
-      return if s3_bucket
+    def mapping_for(type)
+      mappings[type.to_sym]
+    end
 
-      upload_message = <<~MSG
-        The %<identifier>s stack's body is too large to be use the template_body
-        option, and therefore must use the template_url option instead. You can
-        configure Humidifier to do this automatically by setting up the s3
-        config on the top-level Humidifier object like so:
+    def stack_path=(stack_path)
+      unless File.exist?(stack_path)
+        raise Error, "Invalid filepath: #{stack_path}"
+      end
 
-            Humidifier.configure do |config|
-              config.s3_bucket = 'my.s3.bucket'
-              config.s3_prefix = 'my-prefix/' # optional
-            end
-      MSG
+      @stack_path = stack_path
+    end
 
-      raise upload_message.gsub('%<identifier>s', identifier)
+    def stack_names
+      Dir["#{stack_path}/*"].each_with_object([]) do |name, names|
+        names << File.basename(name) if File.directory?(name)
+      end
     end
-    # rubocop:enable Metrics/MethodLength
+
+    private
+
+    attr_reader :mappings
   end
 end

data/lib/humidifier/directory.rb

@@ -0,0 +1,132 @@
+# frozen_string_literal: true
+
+module Humidifier
+  # Represents a directory on the filesystem containing YAML files that
+  # correspond to resources belonging to a stack. Contains all of the logic for
+  # interfacing with humidifier to deploy stacks, validate them, display them.
+  class Directory
+    # Represents an exported resource in a stack for use in cross-stack
+    # references.
+    Export =
+      Struct.new(:name, :attribute) do
+        def value
+          if attribute.is_a?(String)
+            Humidifier.fn.get_att([name, attribute])
+          else
+            Humidifier.ref(name)
+          end
+        end
+      end
+
+    attr_reader :name, :pattern, :prefix, :exports, :stack_name
+
+    def initialize(name, pattern: nil, prefix: nil)
+      @name       = name
+      @pattern    = pattern
+      @prefix     = prefix
+      @exports    = []
+      @stack_name = "#{prefix || Humidifier.config.stack_prefix}#{name}"
+    end
+
+    def create_change_set
+      return unless valid?
+
+      stack.create_change_set(
+        capabilities: %w[CAPABILITY_IAM CAPABILITY_NAMED_IAM]
+      )
+    end
+
+    def deploy(wait = false, parameter_values = {})
+      return unless valid?
+
+      stack.public_send(
+        wait ? :deploy_and_wait : :deploy,
+        capabilities: %w[CAPABILITY_IAM CAPABILITY_NAMED_IAM],
+        parameters: parameter_values
+      )
+    end
+
+    def to_cf
+      stack.to_cf
+    end
+
+    def upload
+      stack.upload if valid?
+    end
+
+    def valid?
+      stack.valid?
+    end
+
+    private
+
+    def stack
+      Stack.new(
+        name: stack_name,
+        description: "Resources for #{stack_name}",
+        resources: resources,
+        outputs: outputs,
+        parameters: parameters
+      )
+    end
+
+    def outputs
+      exports.each_with_object({}) do |export, exported|
+        exported[export.name] =
+          Output.new(value: export.value, export_name: export.name)
+      end
+    end
+
+    def parameters
+      @parameters ||=
+        begin
+          parameter_filepath =
+            Humidifier.config.files_for(name).detect do |filepath|
+              File.basename(filepath, '.yml') == 'parameters'
+            end
+
+          parameter_filepath ? parameters_from(parameter_filepath) : {}
+        end
+    end
+
+    def parameters_from(filepath)
+      loaded = YAML.load_file(filepath)
+      return {} unless loaded
+
+      loaded.each_with_object({}) do |(name, opts), params|
+        opts = opts.map { |key, value| [key.to_sym, value] }.to_h
+        params[name] = Parameter.new(opts)
+      end
+    end
+
+    def parse(filepath, type)
+      mapping = Humidifier.config.mapping_for(type)
+      return {} if mapping.nil?
+
+      loaded = YAML.load_file(filepath)
+      return {} unless loaded
+
+      loaded.each_with_object({}) do |(name, attributes), resources|
+        next if pattern && name !~ pattern
+
+        attribute = attributes.delete('export')
+        exports << Export.new(name, attribute) if attribute
+
+        resources[name] = mapping.resource_for(name, attributes)
+      end
+    end
+
+    def resources
+      filepaths = Humidifier.config.files_for(name)
+
+      filepaths.each_with_object({}) do |filepath, resources|
+        basename = File.basename(filepath, '.yml')
+
+        # Explicitly skip past parameters so we can pull them out later
+        next if basename == 'parameters'
+
+        resources.merge!(parse(filepath, basename))
+      end
+    end
+  end
+end

data/lib/humidifier/stack.rb

@@ -3,7 +3,13 @@
 module Humidifier
   # Represents a CFN stack
   class Stack
-    class TemplateTooLargeError < StandardError
+    class NoResourcesError < Error
+      def initialize(stack, action)
+        super("Refusing to #{action} stack #{stack.name} with no resources")
+      end
+    end
+
+    class TemplateTooLargeError < Error
       def initialize(bytesize)
         super(
           "Cannot use a template > #{MAX_TEMPLATE_URL_SIZE} bytes " \
@@ -14,6 +20,22 @@ module Humidifier
       end
     end
 
+    class UploadNotConfiguredError < Error
+      def initialize(identifier)
+        super(<<~MSG)
+          The #{identifier} stack's body is too large to be use the
+          template_body option, and therefore must use the template_url option
+          instead. You can configure Humidifier to do this automatically by
+          setting up the s3 config on the top-level Humidifier object like so:
+
+              Humidifier.configure do |config|
+                config.s3_bucket = 'my.s3.bucket'
+                config.s3_prefix = 'my-prefix/' # optional
+              end
+        MSG
+      end
+    end
+
     # The AWS region, can be set through the environment, defaults to us-east-1
     AWS_REGION = ENV['AWS_REGION'] || 'us-east-1'
 
@@ -109,6 +131,8 @@ module Humidifier
     end
 
     def create_change_set(opts = {})
+      raise NoResourcesError.new(self, :change) unless resources.any?
+
       params = {
         stack_name: identifier,
         change_set_name: "changeset-#{Time.now.strftime('%Y-%m-%d-%H-%M-%S')}"
@@ -128,6 +152,8 @@ module Humidifier
     end
 
     def deploy(opts = {})
+      raise NoResourcesError.new(self, :deploy) unless resources.any?
+
       exists? ? update(opts) : create(opts)
     end
 
@@ -144,8 +170,12 @@ module Humidifier
     end
 
     def update(opts = {})
-      params =
-        { stack_name: identifier }.merge!(template_for(opts)).merge!(opts)
+      params = {
+        capabilities: %w[CAPABILITY_IAM CAPABILITY_NAMED_IAM],
+        stack_name: identifier
+      }
+
+      params.merge!(template_for(opts)).merge!(opts)
 
       try_valid { client.update_stack(params) }
     end
@@ -154,15 +184,29 @@ module Humidifier
       perform_and_wait(:update, opts)
     end
 
-    def upload
-      Humidifier.config.ensure_upload_configured!(identifier)
-      upload_object("#{Humidifier.config.s3_prefix}#{identifier}.json")
+    def upload # rubocop:disable Metrics/AbcSize
+      raise NoResourcesError.new(self, :upload) unless resources.any?
+
+      bucket = Humidifier.config.s3_bucket
+      raise UploadNotConfiguredError, identifier unless bucket
+
+      Aws.config.update(region: AWS_REGION)
+      key = "#{Humidifier.config.s3_prefix}#{identifier}.json"
+
+      Aws::S3::Client.new.put_object(body: to_cf, bucket: bucket, key: key)
+      Aws::S3::Object.new(bucket, key).presigned_url(:get)
     end
 
     def valid?(opts = {})
       params = template_for(opts).merge!(opts)
 
       try_valid { client.validate_template(params) }
+    rescue Aws::CloudFormation::Errors::AccessDenied
+      raise Error, <<~MSG
+        The authenticated AWS profile does not have the requisite permissions
+        to run this command. Ensure the profile has the
+        "cloudformation:ValidateTemplate" IAM permission.
+      MSG
     end
 
     def self.next_default_identifier
@@ -175,33 +219,12 @@ module Humidifier
 
     attr_reader :default_identifier
 
-    def perform_and_wait(method, opts)
-      public_send(method, opts).tap do
-        signal = :"stack_#{method}_complete"
-
-        client.wait_until(signal, stack_name: identifier) do |waiter|
-          waiter.max_attempts = (opts.delete(:max_wait) || MAX_WAIT) / 5
-          waiter.delay = 5
-        end
+    def bytesize
+      to_cf.bytesize.tap do |size|
+        raise TemplateTooLargeError, size if size > MAX_TEMPLATE_URL_SIZE
       end
     end
 
-    def try_valid
-      yield || true
-    rescue Aws::CloudFormation::Errors::ValidationError => error
-      warn(error.message)
-      warn(error.backtrace)
-      false
-    end
-
-    def upload_object(key)
-      Aws.config.update(region: AWS_REGION)
-      bucket = Humidifier.config.s3_bucket
-
-      Aws::S3::Client.new.put_object(body: to_cf, bucket: bucket, key: key)
-      Aws::S3::Object.new(bucket, key).presigned_url(:get)
-    end
-
     def enumerable_resources
       ENUMERABLE_RESOURCES.each_with_object({}) do |(name, prop), list|
         resources = public_send(prop)
@@ -214,6 +237,17 @@ module Humidifier
       end
     end
 
+    def perform_and_wait(method, opts)
+      public_send(method, opts).tap do
+        signal = :"stack_#{method}_complete"
+
+        client.wait_until(signal, stack_name: identifier) do |waiter|
+          waiter.max_attempts = (opts.delete(:max_wait) || MAX_WAIT) / 5
+          waiter.delay = 5
+        end
+      end
+    end
+
     def static_resources
       STATIC_RESOURCES.each_with_object({}) do |(name, prop), list|
         resource = public_send(prop)
@@ -221,12 +255,6 @@ module Humidifier
       end
     end
 
-    def bytesize
-      to_cf.bytesize.tap do |size|
-        raise TemplateTooLargeError, size if size > MAX_TEMPLATE_URL_SIZE
-      end
-    end
-
     def template_for(opts)
       @template ||=
         if opts.delete(:force_upload) ||
@@ -238,5 +266,13 @@ module Humidifier
           { template_body: to_cf }
         end
     end
+
+    def try_valid
+      yield || true
+    rescue Aws::CloudFormation::Errors::ValidationError => error
+      warn(error.message)
+      warn(error.backtrace)
+      false
+    end
   end
 end

data/lib/humidifier/version.rb

@@ -2,5 +2,5 @@
 
 module Humidifier
   # Gem version
-  VERSION = '3.0.1'
+  VERSION = '3.1.0'
 end

data/lib/humidifier.rb

@@ -8,6 +8,8 @@ require 'yaml'
 require 'aws-sdk-cloudformation'
 require 'aws-sdk-s3'
 require 'fast_underscore'
+require 'thor'
+require 'thor/hollaback'
 
 # Hook into the string extension and ensure it works for certain AWS acronyms
 String.prepend(
@@ -20,6 +22,9 @@ String.prepend(
 
 # container module for all gem classes
 module Humidifier
+  # A parent class for all Humidifier errors for easier rescuing.
+  class Error < StandardError; end
+
   class << self
     # the configuration instance
     def config
@@ -58,18 +63,24 @@ module Humidifier
   end
 end
 
-require 'humidifier/condition'
-require 'humidifier/config'
 require 'humidifier/fn'
+require 'humidifier/ref'
+require 'humidifier/props'
+
+require 'humidifier/cli'
+require 'humidifier/condition'
+require 'humidifier/directory'
 require 'humidifier/loader'
 require 'humidifier/mapping'
 require 'humidifier/output'
 require 'humidifier/parameter'
-require 'humidifier/ref'
 require 'humidifier/resource'
 require 'humidifier/serializer'
 require 'humidifier/stack'
 require 'humidifier/version'
-require 'humidifier/props'
+
+require 'humidifier/config'
+require 'humidifier/config/mapper'
+require 'humidifier/config/mapping'
 
 Humidifier::Loader.load

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: humidifier
 version: !ruby/object:Gem::Version
-  version: 3.0.1
+  version: 3.1.0
 platform: ruby
 authors:
 - Localytics
@@ -52,6 +52,34 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '0.3'
+- !ruby/object:Gem::Dependency
+  name: thor
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.20'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.20'
+- !ruby/object:Gem::Dependency
+  name: thor-hollaback
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.1'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.1'
 - !ruby/object:Gem::Dependency
   name: bundler
   requirement: !ruby/object:Gem::Requirement
@@ -176,8 +204,12 @@ files:
 - LICENSE
 - README.md
 - lib/humidifier.rb
+- lib/humidifier/cli.rb
 - lib/humidifier/condition.rb
 - lib/humidifier/config.rb
+- lib/humidifier/config/mapper.rb
+- lib/humidifier/config/mapping.rb
+- lib/humidifier/directory.rb
 - lib/humidifier/fn.rb
 - lib/humidifier/loader.rb
 - lib/humidifier/mapping.rb