miasma 0.0.1 → 0.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: c544e3078336270c349d518f50150e8d12ef47e4
-  data.tar.gz: 77800023eef8143ba2e034e7f82fc81867594c38
+  metadata.gz: 8f4c2fc901adb9940a31de3c3379646e1225eea6
+  data.tar.gz: c45a82a7de2a1004a51a6c1a3b5fb10c60028088
 SHA512:
-  metadata.gz: 9fc5475434b79aca7f66361ea5b8f75b37640a083fc143645557b60c89348e60e0f970b5571484b15219808a53a44ce289b9e0f3b0c8e2adb56c46282b0a2193
-  data.tar.gz: 947c6d050bf0dccae728ab7dba3754c9a1dc70c7b2d08cf680b195e3a1f72fe93741434750bcbd58e8a4737724825fc57c4f557ddadf70edae43f99cf3ac5cd8
+  metadata.gz: 0e0147bc6672e477a407ff627ea6e37839cae93bc16c6f26499cd5f57a4069358a2e6849062d153bfd1b9940b23fe0568f0e5c6cf40b561be0bdfcd4476ff461
+  data.tar.gz: cdec65223d5ede929aad74fdf520b1a99c9a67a5d63a4559fff2cf414c5bad11eba6342da21fe9a5f27b89ff2f2b71488452e65123da99cac6a7add616902e74

data/CHANGELOG.md

@@ -0,0 +1,2 @@
+# v0.1.0
+* Initial release

data/README.md

@@ -0,0 +1,179 @@
+```
+ ███▄ ▄███▓ ██▓ ▄▄▄        ██████  ███▄ ▄███▓ ▄▄▄
+▓██▒▀█▀ ██▒▓██▒▒████▄    ▒██    ▒ ▓██▒▀█▀ ██▒▒████▄
+▓██    ▓██░▒██▒▒██  ▀█▄  ░ ▓██▄   ▓██    ▓██░▒██  ▀█▄
+▒██    ▒██ ░██░░██▄▄▄▄██   ▒   ██▒▒██    ▒██ ░██▄▄▄▄██
+▒██▒   ░██▒░██░ ▓█   ▓██▒▒██████▒▒▒██▒   ░██▒ ▓█   ▓██▒
+░ ▒░   ░  ░░▓   ▒▒   ▓▒█░▒ ▒▓▒ ▒ ░░ ▒░   ░  ░ ▒▒   ▓▒█░
+░  ░      ░ ▒ ░  ▒   ▒▒ ░░ ░▒  ░ ░░  ░      ░  ▒   ▒▒ ░
+░      ░    ▒ ░  ░   ▒   ░  ░  ░  ░      ░     ░   ▒
+       ░    ░        ░  ░      ░         ░         ░  ░
+```
+
+## Overview
+
+Miasma is YACACL (Yet Another Cloud API Client Library). Instead of
+attempting to cover all the functionalities of all the different
+cloud and virt APIs, miasma is focused on providing a common modeling
+that will "just work". This means there may be many things that seem
+to be missing, but that's probably okay. Consistency is more important
+than overall completeness. Miasma isn't trying to be a replacement
+for libraries like fog, rather it's attempting to supplement those
+libraries by providing a consistent modeling API.
+
+## Usage
+
+### Example
+
+Lets have a look at using the compute model:
+
+```ruby
+compute = Miasma.api(
+  :type => :compute,
+  :provider => :aws,
+  :credentials => {
+    :aws_secret_access_key => 'SECRET',
+    :aws_access_key_id => 'KEY_ID',
+    :aws_region => 'us-west-2'
+  }
+)
+```
+
+With this we can now list existing servers:
+
+```ruby
+compute.servers.all
+```
+
+This will provide an array of `Miasma::Models::Compute::Server`
+instances. It will also cache the result so subsequent calls
+will not require another API call. The list can be refreshed
+by requesting a reload:
+
+```ruby
+compute.servers.reload
+```
+
+### Switching providers
+
+Switching providers requires modification to the API parameters:
+
+```ruby
+compute = Miasma.api(
+  :provider => :rackspace,
+  :credentials => {
+    :rackspace_username => 'USER',
+    :rackspace_api_key => 'KEY',
+    :rackspace_region => 'REGION'
+  }
+)
+```
+
+The `compute` API will act exactly the same as before, now using
+Rackspace instead of AWS.
+
+## Design Objectives
+
+This library is following a few simple objectives:
+
+* Light weight
+* Consistent API
+
+The availabile API is defined first via the models,
+then concrete implementations are built via available
+provider interfaces. This means the provider code is
+structured to support the models instead of the models
+being built around specific providers. The result is
+a clean model interface providing consistency regardless
+of the provider backend.
+
+The "weight" of the library is kept light by using a
+few simple approaches. All code is lazy loaded, so nothing
+will be loaded into the runtime until it is actually required.
+Dependencies are also kept very light, to reduce the number
+of required libraries needing to be loaded. Parser wrapping
+libraries are also used (`multi_json` and `multi_xml`) allowing
+a choice of actual parsing backends in use. This removes
+dependencies on nokogiri unless it's actually desired and
+increases installation speeds.
+
+## Features
+
+### Thin Models
+
+Thin models are a stripped down model that provides a subset
+of information that an actual instance of the model may
+contain. For instance, an `AutoScale::Group` may contain
+a list of `Compute::Server`s. The collection provided within
+the `AutoScale::Group` will be created via the resulting
+API information on the group itself. However, since
+we can provide expected mappings to what API provides
+`Compute` and know these instances will be within the
+`servers` collection, we can use the `#expand` helper to
+automatically load the full instance:
+
+```ruby
+auto_scale = Miasma.api(:type => :auto_scale, :provider ...)
+group = auto_scale.groups.first
+
+# this list will provide the `ThinModel` instances:
+p group.servers.all
+
+# this list will provide the full instances:
+p group.servers.all.map(&:expand)
+```
+
+### Automatic API switching
+
+Resources within a specific provider can span multiple
+API endpoints. To deal with this, the provider API
+implemenetations provide an `#api_for` method which
+will automatically build a new API instance. This
+allows Miasma to hop APIs under the hood to expand
+`ThinModels` as shown above.
+
+## Current status
+
+Miasma is currently under active development and is
+in a beta state. Models are still being implemented
+and spec coverage is following closely behind the
+model completions.
+
+### Currently Supported Providers
+
+* AWS
+* Rackspace
+
+### Models
+
+#### AWS
+
+|Model         |Create|Read|Update|Delete|
+|--------------|------|----|------|------|
+|AutoScale     |  X   |    |      |      |
+|BlockStorage  |      |    |      |      |
+|Compute       |  X   | X  |      |  X   |
+|DNS           |      |    |      |      |
+|LoadBalancer  |  X   | X  |  X   |  X   |
+|Network       |      |    |      |      |
+|Orchestration |  X   | X  |  X   |  X   |
+|Queues        |      |    |      |      |
+|Storage       |      |    |      |      |
+
+#### Rackspace
+
+|Model         |Create|Read|Update|Delete|
+|--------------|------|----|------|------|
+|AutoScale     |  X   |    |      |      |
+|BlockStorage  |      |    |      |      |
+|Compute       |  X   | X  |      |  X   |
+|DNS           |      |    |      |      |
+|LoadBalancer  |      | X  |      |      |
+|Network       |      |    |      |      |
+|Orchestration |  X   | X  |  X   |  X   |
+|Queues        |      |    |      |      |
+|Storage       |      |    |      |      |
+
+## Info
+
+* Repository: https://github.com/chrisroberts/miasma

data/lib/miasma.rb

@@ -0,0 +1,52 @@
+# Load in dependencies
+require 'http'
+require 'multi_json'
+require 'multi_xml'
+
+# Make version available
+require 'miasma/version'
+
+module Miasma
+  autoload :Error, 'miasma/error'
+  autoload :Models, 'miasma/models'
+  autoload :Types, 'miasma/types'
+  autoload :Utils, 'miasma/utils'
+
+  # Generate and API connection
+  #
+  # @param args [Hash]
+  # @option args [String, Symbol] :type API type (:compute, :dns, etc)
+  # @option args [String, Symbol] :provider Service provider
+  # @option args [Hash] :credentials Service provider credentials
+  def self.api(args={})
+    args = Utils::Smash.new(args)
+    [:type, :provider, :credentials].each do |key|
+      unless(args[key])
+        raise ArgumentError.new "Missing required api argument `#{key.inspect}`!"
+      end
+    end
+    args[:type] = Utils.camel(args[:type].to_s).to_sym
+    args[:provider] = Utils.camel(args[:provider].to_s).to_sym
+    begin
+      require "miasma/contrib/#{Utils.snake(args[:provider])}"
+    rescue LoadError
+      # just ignore
+    end
+    base_klass = Models.const_get(args[:type])
+    begin
+      if(base_klass)
+        api_klass = base_klass.const_get(args[:provider])
+        if(api_klass)
+          api_klass.new(args[:credentials].to_smash)
+        else
+          raise Error.new "Failed to locate #{args[:type]} API for #{args[:provider].inspect}"
+        end
+      else
+        raise Error.new "Failed to locate request API type #{args[:type].inspect}"
+      end
+    rescue NameError
+      raise Error.new "Failed to locate request API type #{args[:type].inspect}"
+    end
+  end
+
+end

data/lib/miasma/contrib/aws.rb

@@ -0,0 +1,390 @@
+require 'miasma'
+require 'miasma/utils/smash'
+
+require 'time'
+require 'openssl'
+
+module Miasma
+  module Contrib
+    # Core API for AWS access
+    class AwsApiCore
+
+      module RequestUtils
+
+        # Fetch all results when tokens are being used
+        # for paging results
+        #
+        # @param next_token [String]
+        # @param result_key [Array<String, Symbol>] path to result
+        # @yield block to perform request
+        # @yieldparam options [Hash] request parameters (token information)
+        # @return [Array]
+        def all_result_pages(next_token, *result_key, &block)
+          list = []
+          options = next_token ? Smash.new('NextToken' => next_token) : Smash.new
+          result = block.call(options)
+          content = result.get(*result_key.dup)
+          if(content.is_a?(Array))
+            list += content
+          else
+            list << content
+          end
+          if(token = result.get(:body, 'NextToken'))
+            list += all_result_pages(token, *result_key, &block)
+          end
+          list.compact
+        end
+
+      end
+
+      # @return [String] current time ISO8601 format
+      def self.time_iso8601
+        Time.now.utc.strftime('%Y%m%dT%H%M%SZ')
+      end
+
+      # HMAC helper class
+      class Hmac
+
+        # @return [OpenSSL::Digest]
+        attr_reader :digest
+        # @return [String] secret key
+        attr_reader :key
+
+        # Create new HMAC helper
+        #
+        # @param kind [String] digest type (sha1, sha256, sha512, etc)
+        # @param key [String] secret key
+        # @return [self]
+        def initialize(kind, key)
+          @digest = OpenSSL::Digest.new(kind)
+          @key = key
+        end
+
+        # @return [String]
+        def to_s
+          "Hmac#{digest.name}"
+        end
+
+        # Generate the hexdigest of the content
+        #
+        # @param content [String] content to digest
+        # @return [String] hashed result
+        def hexdigest_of(content)
+          digest << content
+          hash = digest.hexdigest
+          digest.reset
+          hash
+        end
+
+        # Sign the given data
+        #
+        # @param data [String]
+        # @param key_override [Object]
+        # @return [Object] signature
+        def sign(data, key_override=nil)
+          result = OpenSSL::HMAC.digest(digest, key_override || key, data)
+          digest.reset
+          result
+        end
+
+        # Sign the given data and return hexdigest
+        #
+        # @param data [String]
+        # @param key_override [Object]
+        # @return [String] hex encoded signature
+        def hex_sign(data, key_override=nil)
+          result = OpenSSL::HMAC.hexdigest(digest, key_override || key, data)
+          digest.reset
+          result
+        end
+
+      end
+
+      # Base signature class
+      class Signature
+
+        # Create new instance
+        def initialize(*args)
+          raise NotImplementedError.new 'This class should not be used directly!'
+        end
+
+        # Generate the signature
+        #
+        # @param http_method [Symbol] HTTP request method
+        # @param path [String] request path
+        # @param opts [Hash] request options
+        # @return [String] signature
+        def generate(http_method, path, opts={})
+          raise NotImplementedError
+        end
+
+        # URL string escape compatible with AWS requirements
+        #
+        # @param string [String] string to escape
+        # @return [String] escaped string
+        def safe_escape(string)
+          string.to_s.gsub(/([^a-zA-Z0-9_.\-~])/) do
+            '%' << $1.unpack('H2' * $1.bytesize).join('%').upcase
+          end
+        end
+
+      end
+
+      # AWS signature version 4
+      class SignatureV4 < Signature
+
+        # @return [Hmac]
+        attr_reader :hmac
+        # @return [String] access key
+        attr_reader :access_key
+        # @return [String] region
+        attr_reader :region
+        # @return [String] service
+        attr_reader :service
+
+        # Create new signature generator
+        #
+        # @param access_key [String]
+        # @param secret_key [String]
+        # @param region [String]
+        # @param service [String]
+        # @return [self]
+        def initialize(access_key, secret_key, region, service)
+          @hmac = Hmac.new('sha256', secret_key)
+          @access_key = access_key
+          @region = region
+          @service = service
+        end
+
+        # Generate the signature
+        #
+        # @param http_method [Symbol] HTTP request method
+        # @param path [String] request path
+        # @param opts [Hash] request options
+        # @return [String] signature
+        def generate(http_method, path, opts)
+          to_sign = [
+            algorithm,
+            AwsApiCore.time_iso8601,
+            credential_scope,
+            hashed_canonical_request(
+              build_canonical_request(http_method, path, opts)
+            )
+          ].join("\n")
+          signature = sign_request(to_sign)
+          "#{algorithm} Credential=#{access_key}/#{credential_scope}, SignedHeaders=#{signed_headers(opts[:headers])}, Signature=#{signature}"
+        end
+
+        # Sign the request
+        #
+        # @param request [String] request to sign
+        # @return [String] signature
+        def sign_request(request)
+          key = hmac.sign(
+            'aws4_request',
+            hmac.sign(
+              service,
+              hmac.sign(
+                region,
+                hmac.sign(
+                  Time.now.utc.strftime('%Y%m%d'),
+                  "AWS4#{hmac.key}"
+                )
+              )
+            )
+          )
+          hmac.hex_sign(request, key)
+        end
+
+        # @return [String] signature algorithm
+        def algorithm
+          'AWS4-HMAC-SHA256'
+        end
+
+        # @return [String] credential scope for request
+        def credential_scope
+          [
+            Time.now.utc.strftime('%Y%m%d'),
+            region,
+            service,
+            'aws4_request'
+          ].join('/')
+        end
+
+        # Generate the hash of the canonical request
+        #
+        # @param request [String] canonical request string
+        # @return [String] hashed canonical request
+        def hashed_canonical_request(request)
+          hmac.hexdigest_of(request)
+        end
+
+        # Build the canonical request string used for signing
+        #
+        # @param http_method [Symbol] HTTP request method
+        # @param path [String] request path
+        # @param opts [Hash] request options
+        # @return [String] canonical request string
+        def build_canonical_request(http_method, path, opts)
+          [
+            http_method.to_s.upcase,
+            path,
+            canonical_query(opts[:params]),
+            canonical_headers(opts[:headers]),
+            signed_headers(opts[:headers]),
+            canonical_payload(opts)
+          ].join("\n")
+        end
+
+        # Build the canonical query string used for signing
+        #
+        # @param params [Hash] query params
+        # @return [String] canonical query string
+        def canonical_query(params)
+          params ||= {}
+          params = Hash[params.sort_by(&:first)]
+          query = params.map do |key, value|
+            "#{safe_escape(key)}=#{safe_escape(value)}"
+          end.join('&')
+        end
+
+        # Build the canonical header string used for signing
+        #
+        # @param headers [Hash] request headers
+        # @return [String] canonical headers string
+        def canonical_headers(headers)
+          headers ||= {}
+          headers = Hash[headers.sort_by(&:first)]
+          headers.map do |key, value|
+            [key.downcase, value.chomp].join(':')
+          end.join("\n") << "\n"
+        end
+
+        # List of headers included in signature
+        #
+        # @param headers [Hash] request headers
+        # @return [String] header list
+        def signed_headers(headers)
+          headers ||= {}
+          headers.sort_by(&:first).map(&:first).
+            map(&:downcase).join(';')
+        end
+
+        # Build the canonical payload string used for signing
+        #
+        # @param options [Hash] request options
+        # @return [String] body checksum
+        def canonical_payload(options)
+          body = options.fetch(:body, '')
+          if(options[:json])
+            body = MultiJson.dump(options[:json])
+          elsif(options[:form])
+            body = URI.encode_www_form(options[:form])
+          end
+          hmac.hexdigest_of(body)
+        end
+
+      end
+
+      module ApiCommon
+
+        def self.included(klass)
+          klass.class_eval do
+            attribute :aws_access_key_id, String, :required => true
+            attribute :aws_secret_access_key, String, :required => true
+            attribute :aws_region, String, :required => true
+            attribute :aws_host, String
+
+            # @return [Contrib::AwsApiCore::SignatureV4]
+            attr_reader :signer
+          end
+        end
+
+        # Build new API for specified type using current provider / creds
+        #
+        # @param type [Symbol] api type
+        # @return [Api]
+        def api_for(type)
+          memoize(type) do
+            creds = attributes.dup
+            creds.delete(:aws_host)
+            Miasma.api(
+              Smash.new(
+                :type => type,
+                :provider => provider,
+                :credentials => creds
+              )
+            )
+          end
+        end
+
+        # Setup for API connections
+        def connect
+          unless(aws_host)
+            self.aws_host = [
+              self.class::API_SERVICE.downcase,
+              aws_region,
+              'amazonaws.com'
+            ].join('.')
+          end
+          @signer = Contrib::AwsApiCore::SignatureV4.new(
+            aws_access_key_id, aws_secret_access_key, aws_region, self.class::API_SERVICE
+          )
+        end
+
+        # @return [HTTP] connection for requests (forces headers)
+        def connection
+          super.with_headers(
+            'Host' => aws_host,
+            'X-Amz-Date' => Contrib::AwsApiCore.time_iso8601
+          )
+        end
+
+        # @return [String] endpoint for request
+        def endpoint
+          "https://#{aws_host}"
+        end
+
+        # Override to inject signature
+        #
+        # @param connection [HTTP]
+        # @param http_method [Symbol]
+        # @param request_args [Array]
+        # @return [HTTP::Response]
+        # @note if http_method is :post, params will be automatically
+        #  removed and placed into :form
+        def make_request(connection, http_method, request_args)
+          dest, options = request_args
+          path = URI.parse(dest).path
+          options = options.to_smash
+          options[:params] = options.fetch(:params, Smash.new).to_smash.deep_merge('Version' => self.class::API_VERSION)
+          if(http_method.to_sym == :post)
+            if(options[:form])
+              options[:form].merge(options.delete(:params))
+            else
+              options[:form] = options.delete(:params)
+            end
+          end
+          signature = signer.generate(
+            http_method, path, options.merge(
+              Smash.new(
+                :headers => Smash[
+                  connection.default_headers.to_a
+                ]
+              )
+            )
+          )
+          options = Hash[options.map{|k,v|[k.to_sym,v]}]
+          connection.auth(signature).send(http_method, dest, options)
+        end
+
+      end
+
+    end
+  end
+
+  Models::Compute.autoload :Aws, 'miasma/contrib/aws/compute'
+  Models::LoadBalancer.autoload :Aws, 'miasma/contrib/aws/load_balancer'
+  Models::AutoScale.autoload :Aws, 'miasma/contrib/aws/auto_scale'
+  Models::Orchestration.autoload :Aws, 'miasma/contrib/aws/orchestration'
+end