right_aws 1.4.3 → 1.5.0

data/History.txt

@@ -72,3 +72,41 @@ Initial release.
 * r1983, konstantin, 2007-10-25 22:33:00 +0400
   * Fixed ActiveSupport requirement bug (thanks to Toby)
   * Fixed HttpConnection logging to stdout bug (thanks to Toby)
+
+== 1.4.4
+
+* r1999, tve, 2007-11-01 00:07:00 -0700
+  * Fixed escaping issue affecting key names in S3 gem
+  * Fixed duplicate marker in S3 incremental bucket listing
+
+* r2001, konstantin, 2007-11-01 12:03:13  +0300
+  * Fixed multiple permissions assignment on Grantee#grant/revoke
+  * Fixed new grantee permissions set ingnore (Grantee#apply)
+  * S3::Grantee#exists? method added 
+
+* r2109, konstantin, 2007-11-12 21:49:36  +0300
+  * RightAwsBaseInterface: caching implemented.
+    (The Ec2 functions are being cached: describe_images, describe_instances, 
+    describe_security_groups and describe_key_pairs)
+
+== 1.4.5
+== 1.4.6
+*r 2619, konstantin, 01-17-08 16:18:36  +0300
+  * S3 Location constraints support added.
+  * Fixed bug with trailing '/' in the bucket name for 'EU' located buckets
+  * Added: S3Interface#bucket_location, S3::Bucket#location
+
+== 1.4.7
+*r 2622, konstantin, 01-18-08 13:52:20  +0300
+  * Virtual domains doc added
+  * S3 Query API fixed to support virtual domains.
+
+== 1.4.8
+*r 2650, konstantin, 01-24-08 11:12:00  +0300
+  * net_fix.rb moved to right_http_connection
+
+== 1.5.0
+*r 2688, konstantin, 02-30-08 15:42:00  +0300
+  * SDB support added.
+  * RightAws::S3::bucket and RightAws::S3::Bucket.create methods behaviour 
+    changed: param +create+ is set to +false+ by default.

data/Manifest.txt

@@ -9,6 +9,7 @@ lib/awsbase/right_awsbase.rb
 lib/ec2/right_ec2.rb
 lib/s3/right_s3.rb
 lib/s3/right_s3_interface.rb
+lib/sdb/right_sdb_interface.rb
 lib/sqs/right_sqs.rb
 lib/sqs/right_sqs_interface.rb
 test/ec2/test_helper.rb
@@ -16,6 +17,8 @@ test/ec2/test_right_ec2.rb
 test/s3/test_helper.rb
 test/s3/test_right_s3.rb
 test/s3/test_right_s3_stubbed.rb
+test/sdb/test_right_sdb.rb
+test/sdb/test_helper.rb
 test/sqs/test_helper.rb
 test/sqs/test_right_sqs.rb
 test/ts_right_aws.rb

data/README.txt

@@ -5,15 +5,16 @@ For information about RightScale, see http://www.rightscale.com
 
 == DESCRIPTION:
 
-The RightScale AWS gems have been designed to provide a robust, fast, and secure interface to Amazon EC2, Amazon, S3, and Amazon SQS. These gems have been used in production by RightScale since late 2006 and are being maintained to track enhancements made by Amazon. The RightScale AWS gems comprise:
+The RightScale AWS gems have been designed to provide a robust, fast, and secure interface to Amazon EC2, Amazon S3, Amazon SQS, and Amazon SDB. These gems have been used in production by RightScale since late 2006 and are being maintained to track enhancements made by Amazon. The RightScale AWS gems comprise:
 
 - RightAws::Ec2 -- interface to Amazon EC2 (Elastic Compute Cloud)
 - RightAws::S3 and RightAws::S3Interface -- interface to Amazon S3 (Simple Storage Service)
 - RightAws::Sqs and RightAws::SqsInterface -- interface to Amazon SQS (Simple Queue Service)
+- RightAws::SdbInterface -- interface to Amazon SDB (SimpleDB)
 
 == FEATURES:
 
-- Full programmmatic access to EC2, S3, and SQS.
+- Full programmmatic access to EC2, S3, SQS, and SDB.
 - Complete error handling: all operations check for errors and report complete
   error information by raising an AwsError.
 - Persistent HTTP connections with robust network-level retry layer using
@@ -47,7 +48,7 @@ threads, you may want or need an HTTP connection per thread to enable
 concurrent requests to AWS. The way this plays out in practice is:
 1. If you have a non-multithreaded Ruby program, use the non-multithreaded setting.
 2. If you have a multi-threaded Ruby program, use the multithreaded setting to enable
-   concurrent requests to S3 (or SQS, or EC2).
+   concurrent requests to S3 (or SQS, or SDB, or EC2).
 3. For running under Mongrel/Rails, use the non-multithreaded setting even though
    mongrel is multithreaded.  This is because only one Rails handler is invoked at
    time (i.e. it acts like a single-threaded program)
@@ -55,7 +56,7 @@ concurrent requests to AWS. The way this plays out in practice is:
 Note that due to limitations in the I/O of the Ruby interpreter you
 may not get the degree of parallelism you may expect with the multi-threaded setting.
 
-By default, EC2/S3/SQS interface instances are created in single-threaded mode.  Set
+By default, EC2/S3/SQS/SDB interface instances are created in single-threaded mode.  Set
 "params[:multi_thread]" to "true" in the initialization arguments to use
 multithreaded mode.
 
@@ -67,10 +68,26 @@ multithreaded mode.
   http://developer.amazonwebservices.com/connect/kbcategory.jspa?categoryID=48
 * For SQS read RightAws::Sqs and consult the Amazon SQS API documentation at
   http://developer.amazonwebservices.com/connect/kbcategory.jspa?categoryID=31
+* For SDB read RightAws::Sdb and consult the Amazon SDB API documentation at
+  http://developer.amazonwebservices.com/connect/kbcategory.jspa?categoryID=141
 
 == KNOWN ISSUES:
 
-- Amazon recently (8/07) changed the semantics of the SQS service.  A
+- 2/11/08: If you use RightAws in conjunction with attachment_fu, the
+  right_aws gem must be included (using the require statement) AFTER
+  attachment_fu.  If right_aws is loaded before attachment_fu, you'll
+  encounter errors similar to:
+
+  s3.amazonaws.com temporarily unavailable: (wrong number of arguments (5 for 4))
+
+  or
+
+  'incompatible Net::HTTP monkey-patch'
+
+  This is due to a conflict between the right_http_connection gem and another
+  gem required by attachment_fu.
+
+- 8/07: Amazon has changed the semantics of the SQS service.  A
   new queue may not be created within 60 seconds of the destruction of any
   older queue with the same name.  Certain methods of RightAws::Sqs and
   RightAws::SqsInterface will fail with the message:
@@ -91,7 +108,7 @@ sudo gem install
 
 == LICENSE:
 
-Copyright (c) 2007 RightScale, Inc. 
+Copyright (c) 2007-2008 RightScale, Inc. 
 
 Permission is hereby granted, free of charge, to any person obtaining
 a copy of this software and associated documentation files (the

data/Rakefile

@@ -18,7 +18,7 @@ Hoe.new('right_aws', RightAws::VERSION::STRING) do |p|
   p.url = p.paragraphs_of('README.txt', 0).first.split(/\n/)[1..-1]
   p.changes = p.paragraphs_of('History.txt', 0..1).join("\n\n")
   p.remote_rdoc_dir = "/right_aws_gem_doc"
-  p.extra_deps = [['right_http_connection','>= 1.2.0']]
+  p.extra_deps = [['right_http_connection','>= 1.2.1']]
   p.test_globs = testglobs 
 end
 
@@ -59,4 +59,11 @@ task :testec2 do
   require 'test/ec2/test_right_ec2.rb'
 end
 
+desc "Test just the SDB interface"
+task :testsdb do
+  require 'test/test_credentials'
+  TestCredentials.get_credentials
+  require 'test/sdb/test_right_sdb.rb'
+end
+
 # vim: syntax=Ruby

data/lib/awsbase/right_awsbase.rb

@@ -23,7 +23,8 @@
 
 # Test
 module RightAws
-
+  require 'md5'
+  
   class AwsUtils #:nodoc:
     def self.sign(aws_secret_access_key, auth_string)
       Base64.encode64(OpenSSL::HMAC.digest(OpenSSL::Digest::Digest.new("sha1"), aws_secret_access_key, auth_string)).strip
@@ -41,6 +42,9 @@ module RightAws
     end
   end
 
+  class AwsNoChange < RuntimeError
+  end
+  
   class RightAwsBase
 
     # Amazon HTTP Error handling
@@ -70,11 +74,18 @@ module RightAws
     def self.amazon_problems=(problems_list)
       @@amazon_problems = problems_list
     end
-
+    
   end
 
   module RightAwsBaseInterface
-
+    @@caching = false
+    def self.caching
+      @@caching
+    end
+    def self.caching=(caching)
+      @@caching = caching
+    end
+    
       # Current aws_access_key_id
     attr_reader :aws_access_key_id
       # Last HTTP request object
@@ -91,6 +102,8 @@ module RightAws
     attr_accessor :params
       # RightHttpConnection instance
     attr_reader :connection
+      # Cache
+    attr_reader :cache
 
     def init(service_info, aws_access_key_id, aws_secret_access_key, params={}) #:nodoc:
       @params = params
@@ -107,9 +120,51 @@ module RightAws
       @logger = Logger.new(STDOUT)   if !@logger
       @logger.info "New #{self.class.name} using #{@params[:multi_thread] ? 'multi' : 'single'}-threaded mode"
       @error_handler = nil
+      @cache = {}
     end
 
+    # Returns +true+ if the describe_xxx responses are being cached 
+    def caching?
+      @params.key?(:cache) ? @params[:cache] : @@caching
+    end
+    
+    # Check if the aws function response hits the cache or not.
+    # If the cache hits:
+    # - raises an +AwsNoChange+ exception if +do_raise+ == +:raise+.
+    # - returnes parsed response from the cache if it exists or +true+ otherwise.
+    # If the cache miss or the caching is off then returns +false+.
+    def cache_hits?(function, response, do_raise=:raise)
+      result = false
+      if caching?
+        function     = function.to_sym
+        response_md5 = MD5.md5(response).to_s
+        # well, the response is new, reset cache data
+        unless @cache[function] && @cache[function][:response_md5] == response_md5
+          update_cache(function, {:response_md5 => response_md5, 
+                                  :timestamp    => Time.now, 
+                                  :hits         => 0, 
+                                  :parsed       => nil})
+        else
+          # aha, cache hits, update the data and throw an exception if needed
+          @cache[function][:hits] += 1
+          if do_raise == :raise
+            raise(AwsNoChange, "Cache hit: #{function} response has not changed since "+
+                               "#{@cache[function][:timestamp].strftime('%Y-%m-%d %H:%M:%S')}, "+
+                               "hits: #{@cache[function][:hits]}.")
+          else
+            result = @cache[function][:parsed] || true
+          end
+        end
+      end
+      result
+    end
+    
+    def update_cache(function, hash)
+      (@cache[function.to_sym] ||= {}).merge!(hash) if caching?
+    end
+    
     def on_exception(options={:raise=>true, :log=>true}) # :nodoc:
+      raise if $!.is_a?(AwsNoChange)
       AwsError::on_aws_exception(self, options)
     end
     
@@ -253,7 +308,7 @@ module RightAws
     # Used to force logging.
     def self.system_error?(e)
  	    !e.is_a?(self) || e.message =~ /InternalError|InsufficientInstanceCapacity|Unavailable/
- 	  end
+    end
 
   end
 
@@ -316,21 +371,40 @@ module RightAws
     def check(request)  #:nodoc:
       result           = false
       error_found      = false
+      redirect_detected= false
       error_match      = nil
       last_errors_text = ''
       response         = @aws.last_response
-        # log error
+      # log error
       request_text_data = "#{request[:server]}:#{request[:port]}#{request[:request].path}"
-      @aws.logger.warn("##### #{@aws.class.name} returned an error: #{response.code} #{response.message}\n#{response.body} #####")
-      @aws.logger.warn("##### #{@aws.class.name} request: #{request_text_data} ####")
+      # is this a redirect?
+      # yes!
+      if response.is_a?(Net::HTTPRedirection)
+        redirect_detected = true 
+      else
+        # no, it's an error ...
+        @aws.logger.warn("##### #{@aws.class.name} returned an error: #{response.code} #{response.message}\n#{response.body} #####")
+        @aws.logger.warn("##### #{@aws.class.name} request: #{request_text_data} ####")
+      end
         # Check response body: if it is an Amazon XML document or not:
-      if response.body && response.body[/<\?xml/]         # ... it is a xml document
+      if redirect_detected || (response.body && response.body[/<\?xml/])   # ... it is a xml document
         @aws.class.bench_xml.add! do
           error_parser = RightErrorResponseParser.new
           error_parser.parse(response)
           @aws.last_errors     = error_parser.errors
           @aws.last_request_id = error_parser.requestID
           last_errors_text     = @aws.last_errors.flatten.join("\n")
+          # on redirect :
+          if redirect_detected
+            location = response['location']
+            # ... log information and ...
+            @aws.logger.info("##### #{@aws.class.name} redirect requested: #{response.code} #{response.message} #####")
+            @aws.logger.info("##### New location: #{location} #####")
+            # ... fix the connection data
+            request[:server]   = URI.parse(location).host
+            request[:protocol] = URI.parse(location).scheme
+            request[:port]     = URI.parse(location).port
+          end
         end
       else                               # ... it is not a xml document(probably just a html page?)
         @aws.last_errors     = [[response.code, "#{response.message} (#{request_text_data})"]]
@@ -338,29 +412,34 @@ module RightAws
         last_errors_text     = response.message
       end
         # now - check the error
-      @errors_list.each do |error_to_find|
-        if last_errors_text[/#{error_to_find}/i]
-          error_found = true
-          error_match = error_to_find
-          @aws.logger.warn("##### Retry is needed, error pattern match: #{error_to_find} #####")
-          break
+      unless redirect_detected
+        @errors_list.each do |error_to_find|
+          if last_errors_text[/#{error_to_find}/i]
+            error_found = true
+            error_match = error_to_find
+            @aws.logger.warn("##### Retry is needed, error pattern match: #{error_to_find} #####")
+            break
+          end
         end
       end
         # check the time has gone from the first error come
-      if error_found
+      if redirect_detected || error_found
         # Close the connection to the server and recreate a new one. 
         # It may have a chance that one server is a semi-down and reconnection 
         # will help us to connect to the other server 
-        if @close_on_error 
+        if !redirect_detected && @close_on_error
           @aws.connection.finish "#{self.class.name}: error match to pattern '#{error_match}'" 
         end 
                  
         if (Time.now < @stop_at)
           @retries += 1
-          @aws.logger.warn("##### Retry ##{@retries} is being performed. Sleeping for #{@reiteration_delay} sec. Whole time: #{Time.now-@started_at} sec ####")
-          sleep @reiteration_delay
-          
-          @reiteration_delay *= 2
+          unless redirect_detected
+            @aws.logger.warn("##### Retry ##{@retries} is being performed. Sleeping for #{@reiteration_delay} sec. Whole time: #{Time.now-@started_at} sec ####")
+            sleep @reiteration_delay 
+            @reiteration_delay *= 2
+          else
+            @aws.logger.info("##### Retry ##{@retries} is being performed due to a redirect.  ####")
+          end
           result = @aws.request_info(request, @parser)
         else
           @aws.logger.warn("##### Ooops, time is over... ####")
@@ -516,14 +595,27 @@ module RightAws
   #      PARSERS: Errors
   #-----------------------------------------------------------------
 
+#<Error>
+#  <Code>TemporaryRedirect</Code>
+#  <Message>Please re-send this request to the specified temporary endpoint. Continue to use the original request endpoint for future requests.</Message>
+#  <RequestId>FD8D5026D1C5ABA3</RequestId>
+#  <Endpoint>bucket-for-k.s3-external-3.amazonaws.com</Endpoint>
+#  <HostId>ItJy8xPFPli1fq/JR3DzQd3iDvFCRqi1LTRmunEdM1Uf6ZtW2r2kfGPWhRE1vtaU</HostId>
+#  <Bucket>bucket-for-k</Bucket>
+#</Error>
+
   class RightErrorResponseParser < RightAWSParser #:nodoc:
     attr_accessor :errors  # array of hashes: error/message
     attr_accessor :requestID
+#    attr_accessor :endpoint, :host_id, :bucket
     def tagend(name)
       case name
         when 'RequestID' ; @requestID = @text
         when 'Code'      ; @code      = @text
         when 'Message'   ; @message   = @text
+#       when 'Endpoint'  ; @endpoint  = @text
+#       when 'HostId'    ; @host_id   = @text
+#       when 'Bucket'    ; @bucket    = @text
         when 'Error'     ; @errors   << [ @code, @message ]
       end
     end
@@ -531,5 +623,5 @@ module RightAws
       @errors = []
     end
   end
-  
+
 end

data/lib/awsbase/support.rb

@@ -87,6 +87,14 @@ unless defined? ActiveSupport::CoreExtensions
 
   class Hash #:nodoc:
     alias_method :blank?, :empty?
+    
+    # Return a new hash with all keys converted to symbols.
+    def symbolize_keys
+      inject({}) do |options, (key, value)|
+        options[key.to_sym] = value
+        options
+      end
+    end
   end
 
   class String #:nodoc:

data/lib/ec2/right_ec2.rb

@@ -154,19 +154,18 @@ module RightAws
   #-----------------------------------------------------------------
   #-----------------------------------------------------------------
 
-    def ec2_describe_images(type, list, &block) #:nodoc:
+    def ec2_describe_images(type, list) #:nodoc:
       link   = generate_request("DescribeImages", hash_params(type,list.to_a))
       # We do not want to break the logic of parsing hence will use a dummy parser to process all the standart 
       # steps (errors checking etc). The dummy parser does nothig - just returns back the params it received.
-      # Then call the block (if it is) with the response. If the block returns +nil+ or +false+ then do not process
-      # real response parsing and exit with +nil+. Otherwise process parsing and return its result.
+      # If the caching is enabled and hit then throw  AwsNoChange. 
+      # P.S. caching works for the whole images list only! (when the list param is blank)
       response, params = request_info(link, QEc2DummyParser.new)
-      if block
-        return nil unless block.call(response)
-      end
+      # if cache is ON and hits then throws AwsNoChange 
+      cache_hits?(:describe_images, response.body) if list.blank?
       parser = QEc2DescribeImagesParser.new(:logger => @logger)
       @@bench.xml.add!{ parser.parse(response, params) }
-      parser.result.collect do |image|
+      result = parser.result.collect do |image|
         { :aws_id            => image.imageId,
           :aws_location      => image.imageLocation,
           :aws_owner         => image.imageOwnerId,
@@ -174,6 +173,9 @@ module RightAws
           :aws_is_public     => image.isPublic,
           :aws_product_codes => image.productCodes }
       end
+      # put parsed data into cache if the caching is ON
+      update_cache(:describe_images, :parsed => result) if list.blank?
+      result
     rescue Exception
       on_exception
     end
@@ -198,19 +200,8 @@ module RightAws
       #      :aws_location => "marcins_cool_public_images/ubuntu-6.10.manifest.xml",
       #      :aws_is_public => true}]
       #
-      # If a block is given then yields a respose before parsing. 
-      # If the block returns +false+ or +nil+ the response is not parsed
-      # and describe_images returns +nil+.
-      # 
-      #  images = ec2.describe_images do |response|
-      #    # Compare the response to the previous. If they are the same skip parsing - nothing
-      #    # has changed at Amazon. Skpping the parsing will save us a lots of time for the big
-      #    # responses.
-      #    response.body == previous_response_body ? false : true
-      #  end
-      #
-    def describe_images(list=[], &block)
-      ec2_describe_images('ImageId', list, &block)
+    def describe_images(list=[])
+      ec2_describe_images('ImageId', list)
     end
 
       #
@@ -219,8 +210,8 @@ module RightAws
       #  ec2.describe_images_by_owner('522821470517')
       #  ec2.describe_images_by_owner('self')
       #
-    def describe_images_by_owner(list, &block)
-      ec2_describe_images('Owner', list, &block)
+    def describe_images_by_owner(list)
+      ec2_describe_images('Owner', list)
     end
 
       #
@@ -229,8 +220,8 @@ module RightAws
       #  ec2.describe_images_by_executable_by('522821470517')
       #  ec2.describe_images_by_executable_by('self')
       #
-    def describe_images_by_executable_by(list, &block)
-      ec2_describe_images('ExecutableBy', list, &block)
+    def describe_images_by_executable_by(list)
+      ec2_describe_images('ExecutableBy', list)
     end
 
 
@@ -379,7 +370,8 @@ module RightAws
                      :aws_image_id       => instance.imageId,
                      :aws_reason         => reason,
                      :aws_product_codes  => instance.productCodes,
-                     :aws_instance_type  => instance.instanceType}
+                     :aws_instance_type  => instance.instanceType,
+                     :aws_launch_time    => instance.launchTime}
         end
       end
       result
@@ -402,29 +394,21 @@ module RightAws
       #      :ssh_key_name       => "staging",
       #      :aws_groups         => ["default"],
       #      :private_dns_name   => "domU-12-34-67-89-01-C9.usma2.compute.amazonaws.com",
-      #      :aws_instance_type  => "m1.small"}, 
+      #      :aws_instance_type  => "m1.small",
+      #      :aws_launch_time    => "2008-1-1T00:00:00.000Z"},
       #       ..., {...}]
       #
-      # If a block is given then yields a respose before parsing. 
-      # If the block returns +false+ or +nil+ the response is not parsed
-      # and describe_instances returns +nil+.
-      # 
-      #  instances = ec2.describe_instances do |response|
-      #    # Compare the response to the previous. If they are the same skip parsing - nothing
-      #    # has changed at Amazon. Skpping the parsing will save us a lots of time for the big
-      #    # responses.
-      #    response.body == previous_response_body ? false : true
-      #  end
-      #
-    def describe_instances(list=[], &block)
+    def describe_instances(list=[])
       link = generate_request("DescribeInstances", hash_params('InstanceId',list.to_a))
       response, params = request_info(link, QEc2DummyParser.new)
-      if block
-        return nil unless block.call(response)
-      end
+      # check cache
+      cache_hits?(:describe_instances, response.body) if list.blank?
       parser = QEc2DescribeInstancesParser.new(:logger => @logger)
       @@bench.xml.add!{ parser.parse(response, params) }
-      get_desc_instances(parser.result)
+      result = get_desc_instances(parser.result)
+      # update parsed data
+      update_cache(:describe_instances, :parsed => result) if list.blank?
+      result
     rescue Exception
       on_exception
     end
@@ -454,7 +438,8 @@ module RightAws
       #     :ssh_key_name       => "my_awesome_key",
       #     :aws_groups         => ["my_awesome_group"],
       #     :private_dns_name   => "",
-      #     :aws_instance_type  => "m1.small"}]
+      #     :aws_instance_type  => "m1.small",
+      #     :aws_launch_time    => "2008-1-1T00:00:00.000Z"}]
       #
     def run_instances(image_id, min_count, max_count, group_ids, key_name, user_data='',  
                       addressing_type = DEFAULT_ADDRESSING_TYPE,  
@@ -493,7 +478,8 @@ module RightAws
       #     :ssh_key_name       => "my_awesome_key", 
       #     :aws_groups         => ["default"], 
       #     :private_dns_name   => "", 
-      #     :aws_instance_type  => "m1.small"}] 
+      #     :aws_instance_type  => "m1.small",
+      #     :aws_launch_time    => "2008-1-1T00:00:00.000Z"}] 
       #     
     def launch_instances(image_id, lparams={}) 
       defaults = { 
@@ -606,23 +592,11 @@ module RightAws
       #         {:to_port => "443", :protocol => "tcp",  :from_port => "443", :cidr_ips => "0.0.0.0/0"}]},
       #    ..., {...}]
       #
-      # If a block is given then yields a respose before parsing. 
-      # If the block returns +false+ or +nil+ the response is not parsed
-      # and describe_security_groups returns +nil+.
-      # 
-      #  groups = ec2.describe_security_groups do |response|
-      #    # Compare the response to the previous. If they are the same skip parsing - nothing
-      #    # has changed at Amazon. Skpping the parsing will save us a lots of time for the big
-      #    # responses.
-      #    response.body == previous_response_body ? false : true
-      #  end
-      #
-    def describe_security_groups(list=[], &block)
+    def describe_security_groups(list=[])
       link   = generate_request("DescribeSecurityGroups", hash_params('GroupName',list.to_a))
       response, params = request_info(link, QEc2DummyParser.new)
-      if block
-        return nil unless block.call(response)
-      end
+      # check cache
+      cache_hits?(:describe_security_groups, response.body) if list.blank?
       parser = QEc2DescribeSecurityGroupsParser.new(:logger => @logger)
       @@bench.xml.add!{ parser.parse(response, params) }
       
@@ -654,7 +628,9 @@ module RightAws
                    :aws_group_name  => item.groupName, 
                    :aws_description => item.groupDescription,
                    :aws_perms       => perms}
-      end  
+      end
+      # update parsed data
+      update_cache(:describe_security_groups, :parsed => result) if list.blank?
       result
     rescue Exception
       on_exception
@@ -757,29 +733,20 @@ module RightAws
       #     {:aws_fingerprint=> "1e:29:30:47:58:6d:7b:8c:9f:08:11:20:3c:44:52:69:74:80:97:08", :aws_key_name=>"key-2"},
       #      ..., {...} ]
       #
-      # If a block is given then yields a respose before parsing. 
-      # If the block returns +false+ or +nil+ the response is not parsed
-      # and describe_key_pairs returns +nil+.
-      # 
-      #  keys = ec2.describe_key_pairs do |response|
-      #    # Compare the response to the previous. If they are the same skip parsing - nothing
-      #    # has changed at Amazon. Skpping the parsing will save us a lots of time for the big
-      #    # responses.
-      #    response.body == previous_response_body ? false : true
-      #  end
-      #
-    def describe_key_pairs(list=[], &block)
+    def describe_key_pairs(list=[])
       link = generate_request("DescribeKeyPairs", hash_params('KeyName',list.to_a))
       response, params = request_info(link, QEc2DummyParser.new)
-      if block
-        return nil unless block.call(response)
-      end
+      # check cache
+      cache_hits?(:describe_key_pairs, response.body) if list.blank?
       parser = QEc2DescribeKeyPairParser.new(:logger => @logger)
       @@bench.xml.add!{ parser.parse(response, params) }
-      parser.result.collect do |key|
+      result = parser.result.collect do |key|
         { :aws_key_name    => key.keyName,
           :aws_fingerprint => key.keyFingerprint }
       end
+      # update parsed data
+      update_cache(:describe_key_pairs, :parsed => result) if list.blank?
+      result
     rescue Exception
       on_exception
     end
@@ -1044,6 +1011,7 @@ module RightAws
       attr_accessor :amiLaunchIndex
       attr_accessor :productCodes
       attr_accessor :instanceType
+      attr_accessor :launchTime
     end
 
     class QEc2DescribeInstancesType #:nodoc:
@@ -1095,6 +1063,7 @@ module RightAws
             end
           when 'productCode'   ; (@instance.productCodes ||= []) << @text
           when 'instanceType'  ; @instance.instanceType = @text
+	  when 'launchTime'    ; @instance.launchTime = @text
         end
       end
       def reset
@@ -1150,6 +1119,7 @@ module RightAws
           when 'RunInstancesResponse'; @result << @reservation
           when 'productCode'   ; (@instance.productCodes ||= []) << @text
           when 'instanceType'  ; @instance.instanceType = @text
+	  when 'launchTime'    ; @instance.launchTime = @text
         end
       end
       def reset