humidifier 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: fdbf719f5542ef196ffba68323350327492e9259
+  data.tar.gz: c21dc198cc7a5f8d6d249eb7a7f8e6ad697f33bc
+SHA512:
+  metadata.gz: 9be95a4d6a8f9baa91cb70c272508d718cc60aecaaa57531036484107cea0126dd01b59f6d54702696a764e630b67ef3d8db97d5776351d72ac3291cfb316533
+  data.tar.gz: 1ae9dbe8cbead00b4e0ea7f4e7c927944af148d9c4a0399b1502cb6cfa00b74a33adfff1c24a5b4de564e0570f9c2283c6c01cc6c52370b95ef78b20a4c9deb1

data/LICENSE

@@ -0,0 +1,21 @@
+The MIT License
+
+Copyright (c) 2016 Localytics http://www.localytics.com
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,88 @@
+# Humidifier
+
+[![Build Status](https://travis-ci.org/localytics/humidifier.svg?branch=master)](https://travis-ci.org/localytics/humidifier)
+[![Coverage Status](https://coveralls.io/repos/github/localytics/humidifier/badge.svg?branch=master&t=52zybb)](https://coveralls.io/github/localytics/humidifier?branch=master)
+[![Gem Version](https://img.shields.io/gem/v/humidifier.svg?maxAge=2592000)](https://rubygems.org/gems/humidifier)
+
+Humidifier allows you to build AWS CloudFormation (CFN) templates programmatically. CFN stacks and resources are represented as Ruby objects with accessors for all their supported properties. Stacks and resources have `to_cf` methods that allow you to quickly inspect what will be uploaded.
+
+For the full docs, go to [https://localytics.github.io/humidifier/](http://localytics.github.io/humidifier/). For local development instructions, see the [Development](https://localytics.github.io/humidifier/#label-Development) section.
+
+Humidifier is tested with Ruby 2.0.0 and higher.
+
+## Getting started
+
+Stacks are represented by the `Humidifier::Stack` class. You can set any of the top-level JSON attributes through the initializer. Resources are represented by an exact mapping from `AWS` resource names to `Humidifier` resources names (e.g. `AWS::EC2::Instance` becomes `Humidifier::EC2::Instance`). Resources have accessors for each JSON attribute. Each attribute can also be set through the `initialize`, `update`, and `update_attribute` methods.
+
+### Example usage
+
+```ruby
+stack = Humidifier::Stack.new(name: 'Example-Stack', aws_template_format_version: '2010-09-09')
+
+load_balancer = Humidifier::ElasticLoadBalancing::LoadBalancer.new(
+  listeners: [{ LoadBalancerPort: 80, Protocol: 'http', InstancePort: 80, InstanceProtocol: 'http' }]
+)
+load_balancer.scheme = 'internal'
+
+auto_scaling_group = Humidifier::AutoScaling::AutoScalingGroup.new(min_size: '1', max_size: '20')
+auto_scaling_group.update(
+  availability_zones: ['us-east-1a'],
+  load_balancer_names: [Humidifier.ref('LoadBalancer')]
+)
+
+stack.add('LoadBalancer', load_balancer)
+stack.add('AutoScalingGroup', auto_scaling_group)
+stack.deploy_and_wait
+```
+
+### Interfacing with AWS
+
+Once stacks have the appropriate resources, you can query AWS to handle all stack CRUD operations. The operations themselves are intuitively named (i.e. `create`, `update`, `delete`). There are also convenience methods for validating a stack body (`valid?`), checking the existence of a stack (`exists?`), and creating or updating based on existence (`deploy`). The `create`, `update`, `delete`, and `deploy` methods all have `_and_wait` corollaries that will cause the main ruby thread to sleep until the operation is complete.
+
+#### SDK version
+
+Humidifier assumes you have an `aws-sdk` gem installed when you call these operations. It detects the version of the gem you have installed and uses the appropriate API depending on what is available. If Humidifier cannot find any way to use the AWS SDK, it will warn you on every API call and simply return false.
+
+#### CloudFormation functions
+
+You can use CFN intrinsic functions and references using `Humidifier.fn.[name]` and `Humidifier.ref`. Those will build appropriate structures that know how to be dumped to CFN syntax appropriately.
+
+#### Change Sets
+
+Instead of immediately pushing your changes to CloudFormation, Humidifier also supports change sets. Change sets are a powerful feature that allow you to see the changes that will be made before you make them. To read more about change sets see the [announcement article](https://aws.amazon.com/blogs/aws/new-change-sets-for-aws-cloudformation/). To use them in Humidifier, `Stack` has the `create_change_set` and `deploy_change_set` methods. The `create_change_set` method will create a change set on the stack. The `deploy_change_set` method will create a change set if the stack currently exists, and otherwise will create the stack.
+
+### Introspection
+
+To see the template body, you can check the `to_cf` method on stacks, resources, fns, and refs. All of them will output a hash of what will be uploaded (except the stack, which will output a string representation).
+
+Humidifier itself contains a registry of all possible resources that it supports. You can access it with `Humidifier.registry` which is a hash of AWS resource name pointing to the class.
+
+Resources have an `aws_name` method to see how AWS references them. They also contain a `props` method that contains a hash of the name that Humidifier uses to reference the prop pointing to the appropriate prop object.
+
+## Development
+
+To get started, ensure you have ruby installed, version 2.0 or later. From there, install the `bundler` gem: `gem install bundler` and then `bundle install` in the root of the repository.
+
+### Testing
+
+The default rake task runs the tests. Coverage is reported on the command line, and to coveralls.io in CI. Styling is governed by rubocop. The docs are generated with yard. To run all three of these, run:
+
+    $ bundle exec rake
+    $ bundle exec rubocop
+    $ bundle exec rake yard
+
+### Specs
+
+The specs pulled from the CFN docs live under `/specs`. You can update them by running `bin/get-specs`. This script will scrape the docs by going to the [listings page](http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-template-resource-type-ref.html), finding the list of CFN resources, and then downloading the spec for each resource by going to the individual page.
+
+### Extension
+
+The `underscore` string utility method does a lot of the heavy lifting of changing AWS property names over to ruby method names. As such, it's been extracted to a native extension to increase speed and efficiency. To compile it locally run `rake compile`.
+
+### Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/localytics/humidifier.
+
+### License
+
+The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).

data/ext/humidifier/extconf.rb

@@ -0,0 +1,2 @@
+require 'mkmf'
+create_makefile 'humidifier/humidifier'

data/ext/humidifier/humidifier.c

@@ -0,0 +1,93 @@
+#include <humidifier.h>
+
+// copies from the source string to the destination string after passing through a character whitelist filter
+// note that this is exclusively built for AWS::CloudFormation::Interface, so if AWS ever fixes their docs this can
+// be replaced by strcpy
+static void filter(char* dest, const char* source)
+{
+  int source_idx, dest_idx;
+
+  for (source_idx = 0, dest_idx = 0; source[source_idx] != '\0'; source_idx++) {
+    if (source[source_idx] != ':') {
+      dest[dest_idx++] = source[source_idx];
+    }
+  }
+  dest[dest_idx] = '\0';
+}
+
+// takes a substring from underscore_preprocess like EC2T or AWST and converts it to Ec2T or AwsT
+static void format_substring(char* substr, const int substr_idx, const int capitalize)
+{
+  int idx;
+
+  for (idx = 0; idx < substr_idx; idx++) {
+    if (idx == 0) {
+      substr[idx] = toupper(substr[idx]);
+    }
+    else if ((idx != substr_idx - 1) || capitalize == 0) {
+      substr[idx] = tolower(substr[idx]);
+    }
+  }
+}
+
+// finds occurences of EC2Thing or AWSThing and makes them into Ec2Thing and AwsThing so that underscore can be
+// simpler and just look for capitals
+static void preprocess(char* str)
+{
+  char substr[strlen(str)];
+  int idx, substr_idx;
+
+  for (idx = 0, substr_idx = 0; str[idx] != '\0'; idx++) {
+    if (isupper(str[idx])) {
+      substr[substr_idx++] = str[idx];
+
+      if (str[idx + 1] == '\0') {
+        format_substring(substr, substr_idx, 0);
+        memcpy(str + (idx - substr_idx) + 1, substr, substr_idx);
+      }
+    }
+    else if (isdigit(str[idx]) || (substr_idx != 0)) {
+      if (substr_idx != 1) {
+        format_substring(substr, substr_idx, islower(str[idx]));
+        memcpy(str + (idx - substr_idx), substr, substr_idx);
+      }
+      substr_idx = 0;
+    }
+  }
+}
+
+// takes a string and returns a downcased version where capitals are now separated by underscores
+static VALUE underscore(VALUE self, VALUE str)
+{
+  if (TYPE(str) == T_NIL) return Qnil;
+
+  char *str_value = rb_string_value_cstr(&str);
+  char orig_str[strlen(str_value) + 1];
+
+  filter(orig_str, str_value);
+  preprocess(orig_str);
+
+  // manually null-terminating the string because on Fedora for strings of length 16 this breaks otherwise
+  orig_str[strlen(str_value)] = '\0';
+
+  char new_str[strlen(orig_str) * 2];
+  char prev;
+  int orig_idx, new_idx;
+
+  for (orig_idx = 0, new_idx = 0; orig_str[orig_idx] != '\0'; orig_idx++) {
+    if (orig_idx != 0 && isupper(orig_str[orig_idx])) {
+      new_str[new_idx++] = '_';
+    }
+    new_str[new_idx++] = tolower(orig_str[orig_idx]);
+    prev = tolower(orig_str[orig_idx]);
+  }
+
+  return rb_str_new(new_str, new_idx);
+}
+
+void Init_humidifier()
+{
+  VALUE Humidifier = rb_define_module("Humidifier");
+  VALUE Utils = rb_define_module_under(Humidifier, "Utils");
+  rb_define_singleton_method(Utils, "underscore", underscore, 1);
+}

data/ext/humidifier/humidifier.h

@@ -0,0 +1,14 @@
+#ifndef RUBY_HUMIDIFIER
+#define RUBY_HUMIDIFIER
+
+#include <ctype.h>
+#include <ruby.h>
+
+static void filter(char* dest, const char* source);
+static void format_substring(char* substr, const int substr_idx, const int capitalize);
+static void preprocess(char* str);
+static VALUE underscore(VALUE self, VALUE str);
+
+void Init_humidifier();
+
+#endif

data/lib/humidifier/aws_adapters/base.rb

@@ -0,0 +1,64 @@
+module Humidifier
+
+  # A container module for all adapters for the SDK
+  module AwsAdapters
+
+    # The parent class for the adapters for both versions of the SDK
+    class Base
+
+      # Create a CFN stack
+      def create(payload)
+        try_valid do
+          params = { stack_name: payload.name, template_body: payload.to_cf }.merge(payload.options)
+          response = client.create_stack(params)
+          payload.id = response.stack_id
+          response
+        end
+      end
+
+      # Delete a CFN stack
+      def delete(payload)
+        client.delete_stack({ stack_name: payload.identifier }.merge(payload.options))
+        true
+      end
+
+      # Update a CFN stack if it exists, otherwise create it
+      def deploy(payload)
+        exists?(payload) ? update(payload) : create(payload)
+      end
+
+      # Update a CFN stack
+      def update(payload)
+        try_valid do
+          params = { stack_name: payload.identifier, template_body: payload.to_cf }.merge(payload.options)
+          client.update_stack(params)
+        end
+      end
+
+      # Validate a template in CFN
+      def valid?(payload)
+        try_valid { client.validate_template({ template_body: payload.to_cf }.merge(payload.options)) }
+      end
+
+      %i[create delete deploy update].each do |method|
+        define_method(:"#{method}_and_wait") do |payload|
+          perform_and_wait(method, payload)
+        end
+      end
+
+      private
+
+      def client
+        @client ||= base_module::CloudFormation::Client.new(region: AwsShim::REGION)
+      end
+
+      def try_valid
+        yield || true
+      rescue base_module::CloudFormation::Errors::ValidationError => error
+        $stderr.puts error.message
+        $stderr.puts error.backtrace
+        false
+      end
+    end
+  end
+end

data/lib/humidifier/aws_adapters/noop.rb

@@ -0,0 +1,18 @@
+module Humidifier
+  module AwsAdapters
+
+    # An adapter used when neither SDK is loaded
+    class Noop
+
+      # Capture all STACK_METHODS method calls and warn that they will not be run
+      def method_missing(method, *)
+        if AwsShim::STACK_METHODS.include?(method)
+          puts "WARNING: Cannot run #{method} because aws-sdk not loaded."
+          false
+        else
+          super
+        end
+      end
+    end
+  end
+end

data/lib/humidifier/aws_adapters/sdkv1.rb

@@ -0,0 +1,56 @@
+module Humidifier
+  module AwsAdapters
+
+    # The adapter for v1 of aws-sdk
+    class SDKV1 < Base
+
+      # Cannot create change sets in V1
+      def create_change_set(*)
+        unsupported('create change set')
+      end
+
+      # Cannot deploy change sets in V1
+      def deploy_change_set(*)
+        unsupported('deploy change set')
+      end
+
+      # True if the stack exists in CFN
+      def exists?(payload)
+        base_module::CloudFormation::Stack.new(payload.identifier).exists?
+      end
+
+      private
+
+      def base_module
+        AWS
+      end
+
+      def handle_failure(payload)
+        reasons = []
+        client.describe_stack_events(stack_name: payload.identifier).stack_events.each do |event|
+          next unless event.resource_status.include?('FAILED') && event.key?(:resource_status_reason)
+          reasons.unshift(event.resource_status_reason)
+        end
+        raise "#{payload.name} stack failed:\n#{reasons.join("\n")}"
+      end
+
+      def perform_and_wait(method, payload)
+        response = public_send(method, payload)
+
+        aws_stack = nil
+        Sleeper.new(payload.max_wait) do
+          aws_stack = client.describe_stacks(stack_name: payload.identifier).stacks.first
+          !aws_stack.stack_status.end_with?('IN_PROGRESS')
+        end
+
+        handle_failure(payload) if aws_stack.stack_status =~ /(FAILED|ROLLBACK)/
+        response
+      end
+
+      def unsupported(method)
+        puts "WARNING: Cannot #{method} because that functionality is not supported in V1 of aws-sdk."
+        false
+      end
+    end
+  end
+end

data/lib/humidifier/aws_adapters/sdkv2.rb

@@ -0,0 +1,43 @@
+module Humidifier
+  module AwsAdapters
+
+    # The adapter for v2 of aws-sdk
+    class SDKV2 < Base
+
+      # Create a change set in CFN
+      def create_change_set(payload)
+        payload.merge(change_set_name: "changeset-#{Time.now.strftime('%Y-%m-%d-%H-%M-%S')}")
+        params = { stack_name: payload.identifier, template_body: payload.to_cf }.merge(payload.options)
+        try_valid { client.create_change_set(params) }
+      end
+
+      # Create a change set if the stack exists, otherwise create the stack
+      def deploy_change_set(payload)
+        exists?(payload) ? create_change_set(payload) : create(payload)
+      end
+
+      # True if the stack exists in CFN
+      def exists?(payload)
+        base_module::CloudFormation::Stack.new(name: payload.identifier).exists?
+      end
+
+      private
+
+      def base_module
+        Aws
+      end
+
+      def perform_and_wait(method, payload)
+        method = exists?(payload) ? :update : :create if method == :deploy
+        response = public_send(method, payload)
+
+        client.wait_until(:"stack_#{method}_complete", stack_name: payload.identifier) do |waiter|
+          waiter.max_attempts = payload.max_wait / 5
+          waiter.delay = 5
+        end
+
+        response
+      end
+    end
+  end
+end

data/lib/humidifier/aws_shim.rb

@@ -0,0 +1,60 @@
+require 'humidifier/aws_adapters/base'
+require 'humidifier/aws_adapters/noop'
+require 'humidifier/aws_adapters/sdkv1'
+require 'humidifier/aws_adapters/sdkv2'
+
+module Humidifier
+
+  # Optionally provides aws-sdk functionality if the gem is loaded
+  class AwsShim
+
+    # The AWS region, can be set through the environment, defaults to us-east-1
+    REGION = ENV['AWS_REGION'] || 'us-east-1'
+
+    # Methods that are sent over to the aws adapter from the stack
+    STACK_METHODS = %i[
+      create delete deploy exists? update valid?
+      create_and_wait delete_and_wait deploy_and_wait update_and_wait
+      create_change_set deploy_change_set
+    ].freeze
+
+    attr_accessor :shim
+
+    # Attempt to require both aws-sdk-v1 and aws-sdk, then set the shim based on what successfully loaded
+    def initialize
+      try_require_sdk('aws-sdk-v1')
+      try_require_sdk('aws-sdk')
+
+      self.shim =
+        if Object.const_defined?(:AWS)
+          AwsAdapters::SDKV1.new
+        elsif Object.const_defined?(:Aws)
+          AwsAdapters::SDKV2.new
+        else
+          AwsAdapters::Noop.new
+        end
+    end
+
+    class << self
+      extend Forwardable
+      def_delegators :shim, *STACK_METHODS
+
+      # The shim singleton
+      def instance
+        @instance ||= new
+      end
+
+      # The target of all of the forwarding
+      def shim
+        instance.shim
+      end
+    end
+
+    private
+
+    def try_require_sdk(name)
+      require name
+    rescue LoadError # rubocop:disable Lint/HandleExceptions
+    end
+  end
+end

data/lib/humidifier/condition.rb

@@ -0,0 +1,17 @@
+module Humidifier
+
+  # Represents a CFN stack condition
+  class Condition
+
+    attr_accessor :opts
+
+    def initialize(opts)
+      self.opts = opts
+    end
+
+    # CFN stack syntax
+    def to_cf
+      Serializer.dump(opts)
+    end
+  end
+end

data/lib/humidifier/fn.rb

@@ -0,0 +1,28 @@
+module Humidifier
+
+  # Builds CFN function calls
+  class Fn
+
+    # The list of all internal functions provided by AWS from
+    # http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference.html
+    FUNCTIONS = Utils.underscored(%w[And Base64 Equals FindInMap GetAtt GetAZs If Join Not Or Select])
+
+    attr_accessor :name, :value
+
+    def initialize(name, value)
+      self.name = "Fn::#{name}"
+      self.value = value
+    end
+
+    # CFN stack syntax
+    def to_cf
+      { name => value }
+    end
+
+    class << self
+      FUNCTIONS.each do |name, func|
+        define_method(func) { |value| new(name, value) }
+      end
+    end
+  end
+end

data/lib/humidifier/loader.rb

@@ -0,0 +1,53 @@
+module Humidifier
+  # Pre-setting this module because AWS has a "Config" module and the below register method dynamically looks up the
+  # module to see whether or not it exists, which before ruby 2.2 would result in the warning:
+  #   `const_defined?': Use RbConfig instead of obsolete and deprecated Config.
+  # @aws AWS::Config
+  module Config
+  end
+
+  # Reads each of the files under /specs/ and loads them each as a class
+  class Loader
+
+    # loop through the specs and register each class
+    def load
+      spec_directory = File.expand_path(File.join('..', '..', '..', 'specs', '*'), __FILE__)
+      Dir[spec_directory].each { |filepath| load_from(filepath) }
+    end
+
+    # convenience class method to build a new loader and call load
+    def self.load
+      new.load
+    end
+
+    private
+
+    def build_class(aws_name, spec)
+      Class.new(Resource) do
+        self.aws_name = aws_name
+        self.props = spec.each_with_object({}) do |spec_line, props|
+          prop = Props.from(spec_line)
+          props[prop.name] = prop unless prop.name.nil?
+        end
+      end
+    end
+
+    def load_from(filepath)
+      group, resource = Pathname.new(filepath).basename('.cf').to_s.split('-')
+      spec = File.readlines(filepath).select do |line|
+        # flipflop operator (http://stackoverflow.com/questions/14456634)
+        true if line.include?('Properties')...(line.strip == '}')
+      end
+      register(group, resource, spec[1..-2])
+    end
+
+    def register(group, resource, spec)
+      aws_name = "AWS::#{group}::#{resource}"
+      resource_class = build_class(aws_name, spec)
+
+      Humidifier.const_set(group, Module.new) unless Humidifier.const_defined?(group)
+      Humidifier.const_get(group).const_set(resource, resource_class)
+      Humidifier.registry[aws_name] = resource_class
+    end
+  end
+end

data/lib/humidifier/mapping.rb

@@ -0,0 +1,17 @@
+module Humidifier
+
+  # Represents a CFN stack mapping
+  class Mapping
+
+    attr_accessor :opts
+
+    def initialize(opts = {})
+      self.opts = opts
+    end
+
+    # CFN stack syntax
+    def to_cf
+      Serializer.dump(opts)
+    end
+  end
+end

data/lib/humidifier/output.rb

@@ -0,0 +1,20 @@
+module Humidifier
+
+  # Represents a CFN stack output
+  class Output
+
+    attr_accessor :description, :value
+
+    def initialize(opts = {})
+      self.description = opts[:description]
+      self.value       = opts[:value]
+    end
+
+    # CFN stack syntax
+    def to_cf
+      cf = { 'Value' => Serializer.dump(value) }
+      cf['Description'] = description if description
+      cf
+    end
+  end
+end

data/lib/humidifier/parameter.rb

@@ -0,0 +1,27 @@
+module Humidifier
+
+  # Represents a CFN stack parameter
+  class Parameter
+
+    # The allowed properties of all stack parameters
+    PROPERTIES = Utils.underscored(%w[AllowedPattern AllowedValues ConstraintDescription Default Description
+                                      MaxLength MaxValue MinLength MinValue NoEcho])
+
+    attr_accessor :type, *PROPERTIES.values
+
+    def initialize(opts = {})
+      PROPERTIES.values.each { |prop| send(:"#{prop}=", opts[prop]) }
+      self.type = opts.fetch(:type, 'String')
+    end
+
+    # CFN stack syntax
+    def to_cf
+      cf = { 'Type' => type }
+      PROPERTIES.each do |name, prop|
+        val = send(prop)
+        cf[name] = Serializer.dump(val) if val
+      end
+      cf
+    end
+  end
+end