simple_aws 0.0.1a → 0.0.1b

data/README.md

@@ -1,64 +1,141 @@
 SimpleAWS
 =========
 
-A thin, simple, forward compatible Ruby wrapper around the various Amazon AWS API.
+A thin, simple, forward compatible Ruby wrapper around the various Amazon AWS APIs. Unless otherwise mentioned below, this library will wrap every API available service listed on this page: http://aws.amazon.com/documentation/
 
-What? Why?!
+What?! Why?
 -----------
 
-Another Ruby library to talk to Amazon Web Services? Why don't you just use fog
-or aws or aws-sdk or right_aws or any of the myriad of others already in existence?
-Why are you creating yet another one?
+Do we really need another Ruby library to talk to Amazon Web Services? Aren't there enough libraries out there that could just use some more help to make them better? What about [fog](http://fog.io), or [aws-sdk](http://rubygems.org/gems/aws-sdk), or [aws](http://rubygems.org/gems/aws), or [right_aws](http://rubygems.org/gems/right_aws)?
 
-There's a simple, two part answer to this:
+While there are a number of well used libraries, I feel they have all fallen prey to the same two problems, problems that SimpleAWS will not have: complexity and forward incompatibility.
 
 ### Complexity
 
-It's in the name. *Simple* AWS. This library focuses on being the simplest possible way
-to communicate with AWS. There's been a growing trend in the Ruby Library scene to abstract
-and wrap everything possible in ActiveRecord-like objects so the library user doesn't
-have to worry about the details of sending requests and parsing responses from AWS.
-Some make it harder than others to get to the raw requests, but once down there you
-run into the other problem these libraries have.
+Every Ruby AWS library in use today is simply too complex. Every library I've tried to use has ended up hurting my productivity as I often have to dive into the code to find out what the parameter list is, what object I need to work with, or what hash keys map to the actual AWS API parameters. Every library that tries to build an Object Abstraction wrapper on top of AWS's APIs suffers from leaky abstractions, which results in yet more time lost trying to figure out what needs to be called, with what, and what gets returned. Software is supposed to be simple, it's supposed to make your life easier as a developer. I've yet to find an AWS library that does this.
+
+The name SimpleAWS isn't a wish or hope, it's the core philosophy. This library focuses on being a very thin communication layer between your Ruby code and Amazon's AWS APIs. Let SimpleAWS handle the messy communication details so your code can do what it needs to do and you can be more productive doing what you need to do. An AWS library should facilitate using AWS, not abstract it.
+
+It's the Unix philosophy, SimpleAWS does one thing and does it well and nothing else.
 
 ### Forward Compatibility
 
-Yes, not Backward, *Forward* compatibility. SimpleAWS is completely forward compatible to
-any changes AWS makes to it's various APIs. It's well known that Amazon's AWS
-is constantly changing, constantly being updated and added to. Unfortunately, there isn't
-a library out there that lives this truth. Either parameters are hard coded, or response
-values are hard coded, and for all of them the requests themselves are hard built into
-the libraries, making it very hard to work with new API requests.
+Ignoring the complexity argument above, what finally drove me to create this library is the complete lack of forward compatibility in all existing Ruby AWS libraries. Any time I wanted to use a new parameter, new action, or new API, I would need to jump into the library itself and implement the missing pieces. In general, this doesn't sound that bad, and in many cases expected, but not for an API wrapper library.
 
-It's time for a library that evolves with Amazon AWS automatically and refuses to
-get in your way. AWS's API is extremely well documented, consistent, and clean. The libraries
-we use to interact with the API should match these truths as well.
+Amazon constantly updates AWS APIs, adding parameters and actions, and at times entire new APIs. An AWS library should work *with* the API in question not fight against it. The only thing a hard-coded parameter list to a method does is add confusion and more mental disconnects.  SimpleAWS simply says no, it won't force anything on the user. Use the names of the API params and them alone. If a new parameter is added to the API you're using, just use it, SimpleAWS doesn't care, it will just work.
 
-How Simple?
------------
+SimpleAWS is the first and (from what I've found) only Ruby AWS library that is forward compatible with nigh any change Amazon could make to their APIs.
+
+Surely SimpleAWS isn't just `curl`?
+-----------------------------------
+
+It's well know that AWS has its share of warts and wtfs. SimpleAWS doesn't try to fix these, as that's the path towards being yet another over-complicated and rigid library that SimpleAWS is trying to solve.
 
-Open a connection to the interface you want to talk to, and start calling methods.
+What SimpleAWS does do is add some logic to ensure it follows the Principle of Least Surprise.
+
+### Calling
+
+First of all, calling actions are implemented as ruby methods, handled through method_missing. You can call the AWS actions by AWSName or by ruby_name, they both work:
 
 ```ruby
-ec2 = AWS::EC2.new(key, secret)
-response = ec2.describe_instances
+ec2 = AWS::EC2.new key, secret
+ec2.describe_instances
 ```
 
-If this looks familiar to other libraries, well, it's hard to get much simpler than this. Once
-you move past no parameter methods though, the differences abound. What happens when Amazon
-adds another parameter to DescribeInstances? SimpleAWS doesn't care, just start using it.
+or
+
+```ruby
+ec2 = AWS::EC2.new key, secret
+ec2.DescribeInstances
+```
+
+### Parameters
+
+With that in mind, here's what SimpleAWS allows. The following three are equivalent:
+
+#### Just Call It
+
+You can't get simpler than just using the names of the parameters as defined in the AWS docs:
+
+```ruby
+ec2 = AWS::EC2.new key, secret
+ec2.describe_instances({
+  "Filter.1.Name" => "instance-state-name",
+  "Filter.1.Value.1" => "running"
+  "Filter.1.Value.2" => "pending"
+})
+```
+
+#### Use Ruby Arrays
+
+Ruby Arrays will automatically be built into the "Key.n" format you see in the AWS docs:
+
+```ruby
+ec2 = AWS::EC2.new key, secret
+ec2.describe_instances({
+  "Filter.1.Name" => "instance-state-name",
+  "Filter.1.Value" => ["running", "pending"]
+})
+```
+
+#### Use Ruby Hashes even!
+
+You can take this the next step and use a Ruby Hash to make this even cleaner:
+
+```ruby
+ec2 = AWS::EC2.new key, secret
+ec2.describe_instances({
+  "Filter" => [
+    {"Name" => "instance-state-name", "Value" => ["running", "pending"]}
+  ]
+})
+```
+
+### Response Parsing
+
+All requests return an AWS::Response which does a few cleanup tasks on the resulting XML to make it easier to query and to hide some of the worst warts an XML body tends to have.
+
+First of all, whereever AWS returns a list of items, they almost always get wrapped up in an <item> or <member> wrapper tag. Response gets rid of those tags and gives you a flat array to work with.
+
+Second, the resulting XML can have one or two wrapper objects that do nothing but encapsulate the information you're interested in. Response also jumps past these wrapper objects so you have direct access to the data in the response.
+
+All response objects are infinitely deep queryable via methods or Hash access. See the samples, tests, and AWS::Response for more details and examples of usage. At all times you can easily inspect the current Response object for the entire response body, or just the rest of the body at the current depth level.
+
+Implemented APIs
+----------------
+
+These are the following Amazon APIs that SimpleAWS currently handles:
+
+* EC2
+* ELB
+* IAM
+* MapReduce
+* Auto Scaling
+* RDS
+* ElastiCache
+* Elastic Beanstalk
+* CloudFormation
+* SNS
+* CloudWatch
+* Import / Export
+* Mechanical Turk
+* SQS (Simple Queue Service)
+* SES (Simple Email Service)
+
+Yet to be Implemented
+---------------------
+
+* S3
+* CloudFront
 
-SimpleAWS is as light of a Ruby wrapper around the AWS APIs as possible. There are no
-hard-coded parameters, no defined response types, you work directly with what the AWS
-API says you get.
+Currently Out-Of-Scope
+----------------------
 
-What SimpleAWS does do is to hide the communication complexities, the XML parsing, and
-if you want to use it, some of the odd parameter systems AWS uses (PublicIp.n and the like).
-On top of this, SimpleAWS works to ensure everything possible is as Ruby as possible. Methods
-are underscore, the Response object can be queried using methods or a hash structure, and
-parameter keys are converted to CamelCase strings as needed.
+The following API endpoints will not currently be handled in SimpleAWS. These libraries are much more complicated than just a query-and-response API and are related to systems that will most likely need a lot more work than SimpleAWS wants to give. That said if you need these APIs implemented in this library, open an Issue or a Pull Request and we'll work from there.
 
-You're trying to use Amazon AWS, don't let libraries get in your way.
+* FWS (Fulfillment Web Service)
+* FPS & ASP (Flexible Payments Service)
+* Route53
 
 Project Info
 ------------

data/lib/aws/auto_scaling.rb

@@ -0,0 +1,24 @@
+require 'aws/api'
+require 'aws/call_types/action_param'
+require 'aws/signing/version2'
+
+module AWS
+
+  ##
+  # Amazon Auto Scaling
+  #
+  # http://docs.amazonwebservices.com/AutoScaling/latest/APIReference/Welcome.html
+  #
+  # All requests are POST and always through HTTPS. Use the third parameter to
+  # #initialize if you need to talk to a region other than us-east-1.
+  ##
+  class AutoScaling < API
+    endpoint "autoscaling"
+    use_https true
+    version "2011-01-01"
+
+    include CallTypes::ActionParam
+    include Signing::Version2
+  end
+
+end

data/lib/aws/call_types/action_param.rb

@@ -7,7 +7,7 @@ module AWS
 
     ##
     # Implement call handling to work with the ?Action param, signing the message
-    # according to that which is defined in EC2 and ELB.
+    # according to whatever Signing module is included along side this module
     ##
     module ActionParam
       ##
@@ -19,50 +19,25 @@ module AWS
         request.params["Action"] = AWS::Util.camelcase(name.to_s)
 
         if args.any? && args.first.is_a?(Hash)
-          args.first.each do |key, value|
-            request.params[key] = value
-          end
+          insert_params_from request, args.first
         end
 
-        connection = AWS::Connection.new
-        connection.call finish_and_sign_request(request)
+        send_request request
       end
 
       protected
 
-      ##
-      # Build and sign the final request, as per the rules here:
-      # http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/index.html?using-query-api.html
-      ##
-      def finish_and_sign_request(request)
-        request.params.merge!({
-          "AWSAccessKeyId" => self.access_key,
-          "SignatureMethod" => "HmacSHA256",
-          "SignatureVersion" => "2",
-          "Timestamp" => Time.now.utc.strftime("%Y-%m-%dT%H:%M:%SZ"),
-          "Version" => self.version
-        })
-
-        request.params["Signature"] = Base64.encode64(sign_request(request.params.clone)).chomp
-
-        request
+      def send_request(request)
+        connection = AWS::Connection.new
+        connection.call finish_and_sign_request(request)
       end
 
-      def sign_request(params)
-        list = params.map {|k, v| [k, Util.uri_escape(v.to_s)] }
-        list.sort! do |a, b|
-          if a[0] == "AWSAccessKeyId"
-            -1
-          else
-            a[0] <=> b[0]
-          end
+      def insert_params_from(request, args = {})
+        args.each do |key, value|
+          request.params[key] = value
         end
-
-        host = self.uri.gsub(/^http[s]:\/\//,'')
-
-        to_sign = "POST\n#{host}\n/\n#{list.map {|p| p.join("=") }.join("&")}"
-        OpenSSL::HMAC.digest("sha256", self.secret_key, to_sign)
       end
+
     end
   end
 end

data/lib/aws/cloud_formation.rb

@@ -0,0 +1,25 @@
+require 'aws/api'
+require 'aws/call_types/action_param'
+require 'aws/signing/version2'
+
+module AWS
+
+  ##
+  # Amazon's CloudFormation
+  #
+  # http://docs.amazonwebservices.com/AWSCloudFormation/latest/APIReference/
+  #
+  # All requests are POST and always through HTTPS. Use the third parameter to
+  # #initialize if you need to talk to a region other than us-east-1.
+  ##
+  class CloudFormation < API
+    endpoint "cloudformation"
+    use_https true
+    version "2010-05-15"
+    default_region "us-east-1"
+
+    include CallTypes::ActionParam
+    include Signing::Version2
+  end
+
+end

data/lib/aws/cloud_watch.rb

@@ -0,0 +1,25 @@
+require 'aws/api'
+require 'aws/call_types/action_param'
+require 'aws/signing/version2'
+
+module AWS
+
+  ##
+  # Amazon's Cloud Watch
+  #
+  # http://docs.amazonwebservices.com/AmazonCloudWatch/latest/APIReference/Welcome.html
+  #
+  # All requests are POST and always through HTTPS. Use the third parameter to
+  # #initialize if you need to talk to a region other than us-east-1.
+  ##
+  class CloudWatch < API
+    endpoint "monitoring"
+    use_https true
+    version "2010-08-01"
+    default_region "us-east-1"
+
+    include CallTypes::ActionParam
+    include Signing::Version2
+  end
+
+end

data/lib/aws/core/connection.rb

@@ -23,7 +23,8 @@ module AWS
       AWS::Response.new(
         HTTP.send(request.method,
           request.uri,
-          :query => request.params
+          :query => request.params,
+          :headers => request.headers
         )
       )
     end

data/lib/aws/core/request.rb

@@ -35,11 +35,6 @@ module AWS
   #     {"Name" => "ids", "Value" => ["i-1234", "i-8902"]}
   #   ]
   #
-  # If you have just a single Filter, you don't need to wrap it in an array,
-  # Request will do that for you:
-  #
-  #   request.params["Filter"] = {"Name" => "domain", "Value" => "vpc"}
-  #
   # Straight arrays are handled as well:
   #
   #   request.params["InstanceId"] = ["i-1234", "i-8970"]
@@ -61,7 +56,7 @@ module AWS
         when Array
           process_array key, value
         when Hash
-          process_array key, [value]
+          process_hash key, value
         else
           super
         end
@@ -111,6 +106,11 @@ module AWS
     ##
     attr_reader :params
 
+    ##
+    # Hash of headers to send with the request
+    ##
+    attr_reader :headers
+
     ##
     # Set up a new Request for the given +host+ and +path+ using the given
     # http +method+ (:get, :post, :put, :delete).
@@ -118,8 +118,9 @@ module AWS
     def initialize(method, host, path)
       @method = method
       @host = host
-      @path = path
+      self.path = path
       @params = Params.new
+      @headers = {}
     end
 
     ##
@@ -129,5 +130,13 @@ module AWS
       "#{host}#{path}"
     end
 
+    ##
+    # Set path, makes sure that the path is never
+    # nil or an empty string
+    ##
+    def path=(value)
+      @path = (value.nil? || value == "") ? "/" : value
+    end
+
   end
 end

data/lib/aws/core/util.rb

@@ -8,6 +8,8 @@ module AWS
     # Simpler version of ActiveSupport's camelize
     ##
     def self.camelcase(string, lower_first_char = false)
+      return string if string =~ /[A-Z]/
+
       if lower_first_char
         string[0,1].downcase + camelcase(string)[1..-1]
       else

data/lib/aws/ec2.rb

@@ -1,5 +1,6 @@
 require 'aws/api'
 require 'aws/call_types/action_param'
+require 'aws/signing/version2'
 
 module AWS
 
@@ -17,6 +18,7 @@ module AWS
     version "2011-11-01"
 
     include CallTypes::ActionParam
+    include Signing::Version2
   end
 
 end

data/lib/aws/elasti_cache.rb

@@ -0,0 +1,25 @@
+require 'aws/api'
+require 'aws/call_types/action_param'
+require 'aws/signing/version2'
+
+module AWS
+
+  ##
+  # Amazon's ElastiCache
+  #
+  # http://docs.amazonwebservices.com/AmazonElastiCache/latest/APIReference/
+  #
+  # All requests are POST and always through HTTPS. Use the third parameter to
+  # #initialize if you need to talk to a region other than us-east-1.
+  ##
+  class ElastiCache < API
+    endpoint "elasticache"
+    use_https true
+    version "2011-07-15"
+    default_region "us-east-1"
+
+    include CallTypes::ActionParam
+    include Signing::Version2
+  end
+
+end

data/lib/aws/elastic_beanstalk.rb

@@ -0,0 +1,25 @@
+require 'aws/api'
+require 'aws/call_types/action_param'
+require 'aws/signing/version2'
+
+module AWS
+
+  ##
+  # Amazon's Elastic Beanstalk
+  #
+  # http://docs.amazonwebservices.com/elasticbeanstalk/latest/api/
+  #
+  # All requests are POST and always through HTTPS. Use the third parameter to
+  # #initialize if you need to talk to a region other than us-east-1.
+  ##
+  class ElasticBeanstalk < API
+    endpoint "elasticbeanstalk"
+    use_https true
+    version "2010-12-01"
+    default_region "us-east-1"
+
+    include CallTypes::ActionParam
+    include Signing::Version2
+  end
+
+end

data/lib/aws/elb.rb

@@ -1,5 +1,6 @@
 require 'aws/api'
 require 'aws/call_types/action_param'
+require 'aws/signing/version2'
 
 module AWS
 
@@ -17,5 +18,6 @@ module AWS
     version "2011-11-15"
 
     include CallTypes::ActionParam
+    include Signing::Version2
   end
 end

data/lib/aws/iam.rb

@@ -1,5 +1,6 @@
 require 'aws/api'
 require 'aws/call_types/action_param'
+require 'aws/signing/version2'
 
 module AWS
 
@@ -17,6 +18,7 @@ module AWS
     version "2010-05-08"
 
     include CallTypes::ActionParam
+    include Signing::Version2
   end
 
 end

data/lib/aws/import_export.rb

@@ -0,0 +1,28 @@
+require 'aws/api'
+require 'aws/call_types/action_param'
+require 'aws/signing/version2'
+
+module AWS
+
+  ##
+  # Amazon's Import / Export
+  #
+  # http://docs.amazonwebservices.com/AWSImportExport/latest/API/Welcome.html
+  #
+  # All requests are POST and always through HTTPS.
+  # This API does not support region specifiers
+  ##
+  class ImportExport < API
+    endpoint "importexport"
+    use_https true
+    version "2010-06-03"
+
+    def initialize(key, secret)
+      super(key, secret)
+    end
+
+    include CallTypes::ActionParam
+    include Signing::Version2
+  end
+
+end

data/lib/aws/map_reduce.rb

@@ -0,0 +1,24 @@
+require 'aws/api'
+require 'aws/call_types/action_param'
+require 'aws/signing/version2'
+
+module AWS
+
+  ##
+  # Amazon's Elastic MapReduce
+  #
+  # http://docs.amazonwebservices.com/ElasticMapReduce/latest/API/
+  #
+  # All requests are POST and always through HTTPS. Use the third parameter to
+  # #initialize if you need to talk to a region other than us-east-1.
+  ##
+  class MapReduce < API
+    endpoint "elasticmapreduce"
+    use_https true
+    version "2009-03-31"
+
+    include CallTypes::ActionParam
+    include Signing::Version2
+  end
+
+end

data/lib/aws/mechanical_turk.rb

@@ -0,0 +1,57 @@
+require 'aws/api'
+require 'aws/call_types/action_param'
+
+module AWS
+
+  ##
+  # Amazon's Mechanical Turk
+  #
+  # http://docs.amazonwebservices.com/AWSEC2/latest/APIReference/index.html
+  #
+  # All requests are POST and always through HTTPS.
+  # Instead of regions, this API supports normal and sandbox mode. Use the third
+  # parameter of #initialize to specify sandbox mode.
+  #
+  # For a more fleshed out object API for interacting with MechanicalTurk, you should
+  # give rturk a try here: https://github.com/mdp/rturk
+  ##
+  class MechanicalTurk < API
+    endpoint "mechanicalturk"
+    use_https true
+    version "2011-10-01"
+
+    def initialize(key, secret, sandbox = false)
+      super(key, secret, sandbox ? "sandbox" : nil)
+    end
+
+    include CallTypes::ActionParam
+
+    protected
+
+    # Sign the Turk request according to SignatureVersion 0 rules.
+    #
+    # As this is coming from CallTypes::ActionParam we need to fix
+    # the Action param first thing.
+    def finish_and_sign_request(request)
+      request.params["Operation"] = request.params.delete("Action")
+
+      request.params.merge!({
+        "Service" => "AWSMechanicalTurkRequester",
+        "AWSAccessKeyId" => self.access_key,
+        "Timestamp" => Time.now.utc.strftime("%Y-%m-%dT%H:%M:%SZ"),
+        "Version" => self.version
+      })
+
+      request.params["Signature"] = Base64.encode64(sign_request(request.params.clone)).chomp
+
+      request
+    end
+
+    def sign_request(params)
+      to_sign = [params["Service"], params["Operation"], params["Timestamp"]].join ""
+      OpenSSL::HMAC.digest("sha1", self.secret_key, to_sign)
+    end
+
+  end
+
+end

data/lib/aws/rds.rb

@@ -0,0 +1,24 @@
+require 'aws/api'
+require 'aws/call_types/action_param'
+require 'aws/signing/version2'
+
+module AWS
+
+  ##
+  # Amazon's Relational Database Service
+  #
+  # http://docs.amazonwebservices.com/AmazonRDS/latest/APIReference/
+  #
+  # All requests are POST and always through HTTPS. Use the third parameter to
+  # #initialize if you need to talk to a region other than us-east-1.
+  ##
+  class RDS < API
+    endpoint "rds"
+    use_https true
+    version "2011-04-01"
+
+    include CallTypes::ActionParam
+    include Signing::Version2
+  end
+
+end

data/lib/aws/ses.rb

@@ -0,0 +1,29 @@
+require 'aws/api'
+require 'aws/call_types/action_param'
+require 'aws/signing/version3'
+
+module AWS
+
+  ##
+  # Amazon's Simple Email Service
+  #
+  # http://docs.amazonwebservices.com/ses/latest/APIReference/
+  #
+  # All requests are POST and always through HTTPS.
+  ##
+  class SES < API
+    endpoint "email"
+    use_https true
+    version "2010-12-01"
+    default_region "us-east-1"
+
+    # SES only has one HTTP endpoint
+    def initialize(key, secret)
+      super(key, secret)
+    end
+
+    include CallTypes::ActionParam
+    include Signing::Version3
+  end
+
+end