aws 2.4.5 → 2.5.0

data/README.markdown

@@ -2,7 +2,7 @@
 
 A Ruby gem for all Amazon Web Services.
 
-Brought to you by: [![Appoxy](http://www.simpledeployr.com/images/global/appoxy-small.png)](http://www.appoxy.com)
+Brought to you by: [![Appoxy](https://lh5.googleusercontent.com/_-J9DSaseOX8/TX2Bq564w-I/AAAAAAAAxYU/xjeReyoxa8o/s800/appoxy-small%20%282%29.png)](http://www.appoxy.com)
 
 ## Discussion Group
 
@@ -10,7 +10,7 @@ Brought to you by: [![Appoxy](http://www.simpledeployr.com/images/global/appoxy-
 
 ## Documentation
 
-[http://rubydoc.info/gems/aws/](http://rubydoc.info/gems/aws/)
+[Ruby Docs](http://rubydoc.info/gems/aws/2.4.5/frames)
 
 ## Appoxy Amazon Web Services Ruby Gems
 
@@ -20,21 +20,24 @@ Published by [Appoxy LLC](http://www.appoxy.com), under the MIT License. Special
 
     gem install aws
 
+Then `require 'aws'` in your application.
 
 ## DESCRIPTION:
 
-The RightScale AWS gems have been designed to provide a robust, fast, and secure interface to Amazon EC2, EBS, S3, SQS, SDB, and CloudFront.
-These gems have been used in production by RightScale since late 2006 and are being maintained to track enhancements made by Amazon.
-The RightScale AWS gems comprise:
+The AWS gems have been designed to provide a robust, fast, and secure interface to Amazon EC2, EBS, S3, SQS, SDB, and
+CloudFront.
+The AWS gems comprise:
 
 - Aws::Ec2 -- interface to Amazon EC2 (Elastic Compute Cloud) and the associated EBS (Elastic Block Store)
 - Aws::S3 and Aws::S3Interface -- interface to Amazon S3 (Simple Storage Service)
 - Aws::Sqs and Aws::SqsInterface -- interface to Amazon SQS (Simple Queue Service)
-- Aws::SdbInterface and Aws::ActiveSdb -- interface to Amazon SDB (SimpleDB)
+- Aws::SdbInterface -- interface to Amazon SDB (SimpleDB). See [SimpleRecord for an ActiveRecord like gem](https://github.com/appoxy/simple_record).
 - Aws::AcfInterface -- interface to Amazon CloudFront, a content distribution service
 - Aws::ElbInterface -- interface to Amazon Load Balancing service
 - Aws::MonInterface -- interface to Amazon CloudWatch monitoring service
+- Aws::Iam -- for AWS Identity and Access Management
 
+To use a single piece intead of loading all of then, you can require it explicitly for example: `require 'aws/sqs'`.
 
 ## FEATURES:
 
@@ -91,75 +94,6 @@ By default, EC2/S3/SQS/SDB/ACF interface instances are created in per_request mo
 params[:connection_mode] to :per_thread in the initialization arguments to use
 multithreaded mode.
 
-## GETTING STARTED:
-
-* For EC2 read Aws::Ec2 and consult the Amazon EC2 API documentation at
-  http://developer.amazonwebservices.com/connect/kbcategory.jspa?categoryID=87
-* For S3 read Aws::S3 and consult the Amazon S3 API documentation at
-  http://developer.amazonwebservices.com/connect/kbcategory.jspa?categoryID=48
-* For SQS read Aws::Sqs and consult the Amazon SQS API documentation at
-  http://developer.amazonwebservices.com/connect/kbcategory.jspa?categoryID=31
-
-  Amazon's Migration Guide for moving from first to second generation SQS is
-  avalable at:
-  http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1148
-* For SDB read Aws::SdbInterface, Aws::ActiveSdb, and consult the Amazon SDB API documentation at
-  http://developer.amazonwebservices.com/connect/kbcategory.jspa?categoryID=141
-* For CloudFront (ACF) read Aws::AcfInterface and consult the Amazon CloudFront API documentation at
-  http://developer.amazonwebservices.com/connect/kbcategory.jspa?categoryID=213
-
-## KNOWN ISSUES:
-
-- 7/08: A user has reported that uploads of large files on Windows may be broken on some
-  Win platforms due to a buggy File.lstat.size.  Use the following monkey-patch at your own risk, 
-  as it has been proven to break Rails 2.0 on Windows:
-
-    require 'win32/file'
-    class File
-      def lstat
-        self.stat
-      end
-    end
-
-
-- Attempting to use the Gibberish plugin (used by the Beast forum app) 
-  will break right_aws as well as lots of other code.  Gibberish
-  changes the semantics of core Ruby (specifically, the String class) and thus presents a reliability
-  problem for most Ruby programs.
-
-- 2/11/08: If you use Aws in conjunction with attachment_fu, the
-  right_aws gem must be included (using the require statement) AFTER
-  attachment_fu.  If right_aws is loaded before attachment_fu, you'll
-  encounter errors similar to:
-
-  s3.amazonaws.com temporarily unavailable: (wrong number of arguments (5 for 4))
-
-  or
-
-  'incompatible Net::HTTP monkey-patch'
-
-  This is due to a conflict between the right_http_connection gem and another
-  gem required by attachment_fu.  It may be possible to require right_aws (and
-  thus right_http_connection) in the .after_initialize method of the config object in
-  environment.rb (check the docs for Rails::Configuration.after_initialize).
-
-- 8/07: Amazon has changed the semantics of the SQS service.  A
-  new queue may not be created within 60 seconds of the destruction of any
-  older queue with the same name.  Certain methods of Aws::Sqs and
-  Aws::SqsInterface will fail with the message:
-  "AWS.SimpleQueueService.QueueDeletedRecently: You must wait 60 seconds after deleting a queue before you can create another with the same name."
-  
-## REQUIREMENTS:
-
-Aws requires REXML and the http_connection gem.
-If libxml and its Ruby bindings (distributed in the libxml-ruby gem) are
-present, Aws can be configured to use them:
-
-    Aws::AwsParser.xml_lib = 'libxml'
-
-Any error with the libxml installation will result in Aws failing-safe to
-REXML parsing.
-
 
 == LICENSE:
 

data/lib/acf/acf_interface.rb

@@ -88,9 +88,11 @@ module Aws
     end
 
     @@bench          = AwsBenchmarkingBlock.new
- def self.bench
+
+    def self.bench
       @@bench
     end
+
     def self.bench_xml
       @@bench.xml
     end
@@ -132,11 +134,11 @@ module Aws
       headers['content-type'] ||= 'text/xml' if body
       headers['date']          = Time.now.httpdate
       # Auth
-      signature                = AwsUtils::sign(@aws_secret_access_key, headers['date'])
+      signature                = Utils::sign(@aws_secret_access_key, headers['date'])
       headers['Authorization'] = "AWS #{@aws_access_key_id}:#{signature}"
       # Request
       path                     = "#{@params[:default_service]}/#{API_VERSION}/#{path}"
-      request                  = "Net::HTTP::#{method.capitalize}".constantize.new(path)
+      request                  = Net::HTTP.const_get(method.capitalize).new(path)
       request.body = body if body
       # Set request headers
       headers.each { |key, value| request[key.to_s] = value }
@@ -240,7 +242,7 @@ module Aws
       rootElement = streaming ? "StreamingDistributionConfig" : "DistributionConfig"
       # join CNAMES
       cnames_str  = ''
-      unless cnames.blank?
+      unless cnames.nil? || cnames.empty?
         cnames.to_a.each { |cname| cnames_str += "\n           <CNAME>#{cname}</CNAME>" }
       end
       caller_reference ||= generate_call_reference

data/lib/aws.rb

@@ -15,7 +15,6 @@ require 'right_http_connection'
 $:.unshift(File.dirname(__FILE__))
 require 'awsbase/require_relative'
 require 'awsbase/benchmark_fix'
-require 'awsbase/support'
 require 'awsbase/awsbase'
 require 'awsbase/aws_response_array'
 require 'ec2/ec2'
@@ -29,4 +28,6 @@ require 'acf/acf_interface'
 require 'elb/elb_interface'
 require 'rds/rds'
 require 'iam/iam'
+require 'ses/ses'
+
 

data/lib/awsbase/awsbase.rb

@@ -28,7 +28,8 @@ module Aws
   require 'cgi'
   require 'uri'
   require 'xmlsimple'
-  require 'active_support/core_ext'
+  require 'net/http'
+#  require 'active_support/core_ext'
 
   require_relative 'utils'
   require_relative 'errors'
@@ -42,7 +43,7 @@ module Aws
       # Benchmark::Tms instance for service (Ec2, S3, or SQS) access benchmarking.
       @service = Benchmark::Tms.new()
       # Benchmark::Tms instance for XML parsing benchmarking.
-      @xml     = Benchmark::Tms.new()
+      @xml = Benchmark::Tms.new()
     end
   end
 
@@ -55,15 +56,15 @@ module Aws
 
     # Text, if found in an error message returned by AWS, indicates that this may be a transient
     # error. Transient errors are automatically retried with exponential back-off.
-    AMAZON_PROBLEMS   = ['internal service error',
-                         'is currently unavailable',
-                         'no response from',
-                         'Please try again',
-                         'InternalError',
-                         'ServiceUnavailable', #from SQS docs
-                         'Unavailable',
-                         'This application is not currently available',
-                         'InsufficientInstanceCapacity'
+    AMAZON_PROBLEMS = ['internal service error',
+                       'is currently unavailable',
+                       'no response from',
+                       'Please try again',
+                       'InternalError',
+                       'ServiceUnavailable', #from SQS docs
+                       'Unavailable',
+                       'This application is not currently available',
+                       'InsufficientInstanceCapacity'
     ]
     @@amazon_problems = AMAZON_PROBLEMS
     # Returns a list of Amazon service responses which are known to be transient problems.
@@ -91,10 +92,6 @@ module Aws
         @@bench
       end
 
-      def self.bench
-        @@bench
-      end
-
       def self.bench_xml
         @@bench.xml
       end
@@ -138,23 +135,24 @@ module Aws
 
     def init(service_info, aws_access_key_id, aws_secret_access_key, params={}) #:nodoc:
       @params = params
-      raise AwsError.new("AWS access keys are required to operate on #{service_info[:name]}") \
- if aws_access_key_id.blank? || aws_secret_access_key.blank?
-      @aws_access_key_id     = aws_access_key_id
+      if Aws::Utils.blank?(aws_access_key_id) || Aws::Utils.blank?(aws_secret_access_key)
+        raise AwsError.new("AWS access keys are required to operate on #{service_info[:name]}")
+      end
+      @aws_access_key_id = aws_access_key_id
       @aws_secret_access_key = aws_secret_access_key
       # if the endpoint was explicitly defined - then use it
       if @params[:endpoint_url]
-        @params[:server]   = URI.parse(@params[:endpoint_url]).host
-        @params[:port]     = URI.parse(@params[:endpoint_url]).port
-        @params[:service]  = URI.parse(@params[:endpoint_url]).path
+        @params[:server] = URI.parse(@params[:endpoint_url]).host
+        @params[:port] = URI.parse(@params[:endpoint_url]).port
+        @params[:service] = URI.parse(@params[:endpoint_url]).path
         @params[:protocol] = URI.parse(@params[:endpoint_url]).scheme
-        @params[:region]   = nil
+        @params[:region] = nil
       else
         @params[:server] ||= service_info[:default_host]
         @params[:server] = "#{@params[:region]}.#{@params[:server]}" if @params[:region]
-        @params[:port]        ||= service_info[:default_port]
-        @params[:service]     ||= service_info[:default_service]
-        @params[:protocol]    ||= service_info[:default_protocol]
+        @params[:port] ||= service_info[:default_port]
+        @params[:service] ||= service_info[:default_service]
+        @params[:protocol] ||= service_info[:default_protocol]
         @params[:api_version] ||= service_info[:api_version]
       end
       if !@params[:multi_thread].nil? && @params[:connection_mode].nil? # user defined this
@@ -168,19 +166,19 @@ module Aws
       @logger = ::Rails.logger if !@logger && defined?(::Rails.logger)
       @logger = Logger.new(STDOUT) if !@logger
       @logger.info "New #{self.class.name} using #{@params[:connection_mode].to_s}-connection mode"
-      @error_handler     = nil
-      @cache             = {}
-      @signature_version = (params[:signature_version] || DEFAULT_SIGNATURE_VERSION).to_s
+      @error_handler = nil
+      @cache = {}
+      @signature_version = (params[:signature_version] || service_info[:signature_version] || DEFAULT_SIGNATURE_VERSION).to_s
     end
 
     def signed_service_params(aws_secret_access_key, service_hash, http_verb=nil, host=nil, service=nil)
       case signature_version.to_s
         when '0' then
-          AwsUtils::sign_request_v0(aws_secret_access_key, service_hash)
+          Utils::sign_request_v0(aws_secret_access_key, service_hash)
         when '1' then
-          AwsUtils::sign_request_v1(aws_secret_access_key, service_hash)
+          Utils::sign_request_v1(aws_secret_access_key, service_hash)
         when '2' then
-          AwsUtils::sign_request_v2(aws_secret_access_key, service_hash, http_verb, host, service)
+          Utils::sign_request_v2(aws_secret_access_key, service_hash, http_verb, host, service)
         else
           raise AwsError.new("Unknown signature version (#{signature_version.to_s}) requested")
       end
@@ -199,13 +197,25 @@ module Aws
 #            end
       #params_string  = params.to_a.collect{|key,val| key + "=#{CGI::escape(val.to_s)}" }.join("&")
       # prepare service data
-      service      = lib_params[:service]
-#      puts 'service=' + service.to_s
-      service_hash = {"Action"         => action,
+      service = lib_params[:service]
+
+      now = Time.now.getutc
+      service_hash = {"Action" => action,
                       "AWSAccessKeyId" => aws_access_key}
       service_hash.update("Version" => api_version) if api_version
       service_hash.update(user_params)
-      service_params = signed_service_params(aws_secret_key, service_hash, :get, lib_params[:server], lib_params[:service])
+      headers = {}
+      if signature_version == '3'
+        service_hash["Timestamp"] = now.iso8601
+        service_params = escape_params(service_hash)
+        signature, algorithm = Aws::Utils.signature_version3(aws_secret_key, now)
+        headers['X-Amzn-Authorization'] = "AWS3-HTTPS AWSAccessKeyId=#{aws_access_key}, Algorithm=#{algorithm.upcase}, Signature=#{signature}"
+        headers['Date'] = now.httpdate
+      else
+#      puts 'service=' + service.to_s
+        service_params = signed_service_params(aws_secret_key, service_hash, :get, lib_params[:server], lib_params[:service])
+      end
+
       #
       # use POST method if the length of the query string is too large
       # see http://docs.amazonwebservices.com/AmazonSimpleDB/2007-11-07/DeveloperGuide/MakingRESTRequests.html
@@ -214,23 +224,34 @@ module Aws
           # resign the request because HTTP verb is included into signature
           service_params = signed_service_params(aws_secret_key, service_hash, :post, lib_params[:server], service)
         end
-        request                 = Net::HTTP::Post.new(service)
-        request.body            = service_params
+        request = Net::HTTP::Post.new(service)
+        request.body = service_params
         request['Content-Type'] = 'application/x-www-form-urlencoded; charset=utf-8'
       else
         request = Net::HTTP::Get.new("#{service}?#{service_params}")
       end
+      headers.each_pair do |k, v|
+        request[k] = v
+      end
+      puts "header=" + request['X-Amzn-Authorization']
 
       #puts "\n\n --------------- QUERY REQUEST TO AWS -------------- \n\n"
       #puts "#{@params[:service]}?#{service_params}\n\n"
 
       # prepare output hash
-      {:request  => request,
-       :server   => lib_params[:server],
-       :port     => lib_params[:port],
+      {:request => request,
+       :server => lib_params[:server],
+       :port => lib_params[:port],
        :protocol => lib_params[:protocol]}
     end
 
+    def escape_params(service_hash)
+      canonical_string = service_hash.keys.sort.map do |key|
+        "#{Aws::Utils.amz_escape(key)}=#{Aws::Utils.amz_escape(service_hash[key])}"
+      end.join('&')
+      canonical_string
+    end
+
     def get_conn(connection_name, lib_params, logger)
 #            thread = lib_params[:multi_thread] ? Thread.current : Thread.main
 #            thread[connection_name] ||= Rightscale::HttpConnection.new(:exception => Aws::AwsError, :logger => logger)
@@ -239,20 +260,23 @@ module Aws
       http_conn = nil
       conn_mode = lib_params[:connection_mode]
 
-      # Slice all parameters accepted by Rightscale::HttpConnection#new
-      params    = lib_params.slice(
-          :user_agent, :ca_file, :http_connection_retry_count, :http_connection_open_timeout,
-          :http_connection_read_timeout, :http_connection_retry_delay
-      )
-      params.merge!(:exception => AwsError, :logger => logger)
+      params = { :exception => AwsError, :logger => logger }
+      
+      # Adds all parameters accepted by Rightscale::HttpConnection#new
+      [ :user_agent, :ca_file, :http_connection_retry_count, 
+        :http_connection_open_timeout, :http_connection_read_timeout, 
+        :http_connection_retry_delay 
+      ].each do |key|
+        params[key] = lib_params[key] if lib_params.has_key?(key)
+      end
 
       if conn_mode == :per_request
         http_conn = Rightscale::HttpConnection.new(params)
 
       elsif conn_mode == :per_thread || conn_mode == :single
-        thread                  = conn_mode == :per_thread ? Thread.current : Thread.main
+        thread = conn_mode == :per_thread ? Thread.current : Thread.main
         thread[connection_name] ||= Rightscale::HttpConnection.new(params)
-        http_conn               = thread[connection_name]
+        http_conn = thread[connection_name]
 #                ret = request_info_impl(http_conn, bench, request, parser, &block)
       end
       return http_conn
@@ -280,13 +304,13 @@ module Aws
 
 
     def request_info2(request, parser, lib_params, connection_name, logger, bench, options={}, &block) #:nodoc:
-      ret       = nil
+      ret = nil
 #            puts 'OPTIONS=' + options.inspect
       http_conn = get_conn(connection_name, lib_params, logger)
       begin
         # todo: this QueryTimeout retry should go into a SimpleDbErrorHandler, not here
         retry_count = 1
-        count       = 0
+        count = 0
         while count <= retry_count
           puts 'RETRYING QUERY due to QueryTimeout...' if count > 0
           begin
@@ -325,19 +349,19 @@ module Aws
 
       connection = get_conn(connection_name, lib_params, logger)
       begin
-        @last_request  = request[:request]
+        @last_request = request[:request]
         @last_response = nil
 
-        response       = connection.request(request)
+        response = connection.request(request)
         #       puts "response=" + response.body
 #            benchblock.service.add!{ response = connection.request(request) }
         # check response for errors...
         @last_response = response
         if response.is_a?(Net::HTTPSuccess)
-          @error_handler     = nil
+          @error_handler = nil
 #                benchblock.xml.add! { parser.parse(response) }
 #                return parser.result
-          force_array        = params[:force_array] || false
+          force_array = params[:force_array] || false
           # Force_array and group_tags don't work nice together so going to force array manually
           xml_simple_options = {"KeyToSymbol"=>false, 'ForceArray' => false}
           xml_simple_options["GroupTags"] = params[:group_tags] if params[:group_tags]
@@ -358,7 +382,7 @@ module Aws
           parsed = symbolize(parsed, force_array)
 #                puts 'parsed=' + parsed.inspect
           if params[:pull_out_array]
-            ret        = Aws::AwsResponseArray.new(parsed[:response_metadata])
+            ret = Aws::AwsResponseArray.new(parsed[:response_metadata])
             level_hash = parsed
             params[:pull_out_array].each do |x|
               level_hash = level_hash[x]
@@ -373,7 +397,7 @@ module Aws
             end
           elsif params[:pull_out_single]
             # returns a single object
-            ret        = AwsResponseObjectHash.new(parsed[:response_metadata])
+            ret = AwsResponseObjectHash.new(parsed[:response_metadata])
             level_hash = parsed
             params[:pull_out_single].each do |x|
               level_hash = level_hash[x]
@@ -400,6 +424,15 @@ module Aws
 
     end
 
+    # This is the latest and greatest now. Service must have connection_name defined.
+    def request_info_xml_simple3(service_interface, request, options)
+      request_info_xml_simple(service_interface.class.connection_name,
+                              service_interface.params,
+                              request,
+                              service_interface.logger,
+                              options)
+    end
+
     def symbolize(hash, force_array)
       ret = {}
       hash.keys.each do |key|
@@ -412,7 +445,7 @@ module Aws
         elsif val.is_a? Array
           val = val.collect { |x| symbolize(x, force_array) }
         end
-        ret[key.underscore.to_sym] = val
+        ret[Aws::Utils.underscore(key).to_sym] = val
       end
       ret
     end
@@ -430,17 +463,17 @@ module Aws
     def cache_hits?(function, response, do_raise=:raise)
       result = false
       if caching?
-        function     = function.to_sym
+        function = function.to_sym
         # get rid of requestId (this bad boy was added for API 2008-08-08+ and it is uniq for every response)
-        response     = response.sub(%r{<requestId>.+?</requestId>}, '')
+        response = response.sub(%r{<requestId>.+?</requestId>}, '')
         response_md5 =Digest::MD5.hexdigest(response).to_s
         # check for changes
         unless @cache[function] && @cache[function][:response_md5] == response_md5
           # well, the response is new, reset cache data
           update_cache(function, {:response_md5 => response_md5,
-                                  :timestamp    => Time.now,
-                                  :hits         => 0,
-                                  :parsed       => nil})
+                                  :timestamp => Time.now,
+                                  :hits => 0,
+                                  :parsed => nil})
         else
           # aha, cache hits, update the data and throw an exception if needed
           @cache[function][:hits] += 1
@@ -472,10 +505,10 @@ module Aws
 
 
     def request_info_impl(connection, benchblock, request, parser, options={}, &block) #:nodoc:
-      connection    = connection
-      @last_request  = request[:request]
+      connection = connection
+      @last_request = request[:request]
       @last_response = nil
-      response       =nil
+      response =nil
       blockexception = nil
 
 #             puts 'OPTIONS2=' + options.inspect