dmarkov-right_aws 1.10.0

data/History.txt

@@ -0,0 +1,246 @@
+== 1.1.0 2007-08-10
+Initial release.
+
+== 1.2.0 2007-09-12
+
+* r1718, todd, 2007-09-12 15:34:37
+  * # 458, Extensive documentation review, rework, and expansion.  Also added 
+    coverage analysis to the test suite using RCov.
+
+* r1690, todd, 2007-09-07 15:23:11
+  * # 447, Add support.rb to manifest
+
+* r1688, todd, 2007-09-07 13:57:39
+  * # 447, Use Active Support if available, but don't require it.  Load our own
+    extensions if it's not present.  This keeps us from overloading ActiveSupport if
+    a user's already using it.
+
+* r1687, todd, 2007-09-07 11:36:43
+  * # 447, Removed dependency on activesupport
+
+* r1676, konstantin, 2007-09-06 01:27:09
+  * paid AMIs, small fix
+
+* r1667, konstantin, 2007-09-05 12:58:10
+  * # 427, paid AMI support for ec2_instances
+
+* r1658, konstantin, 2007-09-05 01:02:25
+  * params improvements for ec2.describe_xxx, now ones can use a String as well as an Array.
+
+* r1653, konstantin, 2007-09-04 12:31:19
+  * libxml and paid AMI support added
+
+* r1581, tve, 2007-08-24 16:21:45
+  * Improved RightAws documentation
+
+== 1.3.0 2007-09-26
+
+* r1754, todd, 2007-09-19 13:48:34
+  * # 487, # 488, Consolidate a lot of code that was repeated in three places.
+    Fix error handling path when using streaming GET interfaces with S3.
+    Also add a stub for RightHttpConnection which allows more control over the
+    unit tests.  Expand the unit tests and coverage tests. 
+
+* r1755, todd, 2007-09-19 14:29:19
+  * # 487, RDoc fixes after code consolidation
+
+* r1866, konstantin, 2007-10-05 06:17:36
+  * # 220, close connection on HTTP 5xx/4xx errors
+
+== 1.4.0 2007-10-10
+
+* r1868, konstantin, 2007-10-05 06:38:37
+  * # 220, inst_type branch merge into 1.3.0 release
+
+* r1869, konstantin, 2007-10-05 07:32:34
+  * right_http_connection 1.1.1 requirements fixed
+
+* r1879, konstantin, 2007-10-07 02:11:24
+  * # 524, blocks added to ec2_describe_xxx to support aws_cache
+
+* r1924, konstantin, 2007-10-12 11:35:06
+  * # 536, user_data bug fix 
+
+* r1929, tve,  2007-10-15 00:00:11
+  * Fix libxml/rexml selection bug
+
+* r1938, konstantin, 2007-10-16 10:53:56
+  * instance type support is set to default 
+
+== 1.4.3 2007-10-25
+
+* r1983, konstantin, 2007-10-25 22:33:00 +0400
+  * Fixed ActiveSupport requirement bug (thanks to Toby)
+  * Fixed HttpConnection logging to stdout bug (thanks to Toby)
+
+== 1.4.4
+
+* r1999, tve, 2007-11-01 00:07:00 -0700
+  * Fixed escaping issue affecting key names in S3 gem
+  * Fixed duplicate marker in S3 incremental bucket listing
+
+* r2001, konstantin, 2007-11-01 12:03:13  +0300
+  * Fixed multiple permissions assignment on Grantee#grant/revoke
+  * Fixed new grantee permissions set ingnore (Grantee#apply)
+  * S3::Grantee#exists? method added 
+
+* r2109, konstantin, 2007-11-12 21:49:36  +0300
+  * RightAwsBaseInterface: caching implemented.
+    (The Ec2 functions are being cached: describe_images, describe_instances, 
+    describe_security_groups and describe_key_pairs)
+
+== 1.4.5 - 1.4.6
+* r 2619, konstantin, 01-17-08 16:18:36  +0300
+  * S3 Location constraints support added.
+  * Fixed bug with trailing '/' in the bucket name for 'EU' located buckets
+  * Added: S3Interface#bucket_location, S3::Bucket#location
+
+== 1.4.7
+* r 2622, konstantin, 01-18-08 13:52:20  +0300
+  * Virtual domains doc added
+  * S3 Query API fixed to support virtual domains.
+
+== 1.4.8
+* r 2650, konstantin, 01-24-08 11:12:00  +0300
+  * net_fix.rb moved to right_http_connection
+
+== 1.5.0
+* r 2688, konstantin, 02-30-08 15:42:00  +0300
+  * SDB support added.
+  * RightAws::S3::bucket and RightAws::S3::Bucket.create methods behaviour 
+    changed: param +create+ is set to +false+ by default.
+
+== 1.6.0
+
+* r2780, todd, 2008-02-11 11:41:07 -0800 (Mon, 11 Feb 2008), 4 lines
+  * Some doc updates & tweaks: we now support ultra-large PUTs to S3, small SDB
+    cleanups, add SDB to the list of interfaces in RightAws, update some copyrights,
+    warn about loading attachment_fu AFTER right_aws (big no-no).
+
+* r2784, todd, 2008-02-11 13:46:47 -0800 (Mon, 11 Feb 2008), 2 lines
+  * One final clarification: you may get a Net::HTTP bad monkey patch exception
+
+* r2880, todd, 2008-02-25 18:06:22 -0800 (Mon, 25 Feb 2008), 2 lines
+  * Add SQS 'Gen 2' interface implementation
+
+* r2913, todd, 2008-02-29 16:57:07 -0800 (Fri, 29 Feb 2008), 3 lines
+  * SqsGen2 (object interface), unit tests for both interfaces, documentation updates.
+
+* r2922, todd, 2008-03-03 15:26:42 -0800 (Mon, 03 Mar 2008), 2 lines
+  * couple of documentation tweaks in prep for 1.6.0
+
+== 1.6.1
+
+* r2963, todd, 2008-03-06 19:10:23 -0800 (Thu, 06 Mar 2008), 3 lines
+  * (#950) Many minor fixes in incrementally_list_bucket to prevent a death loop
+    in certain rare conditions
+
+== 1.7.0
+
+* r3051, konstantin, 2008-03-14 21:26:12 +0300 (Fri, 14 Mar 2008), 1 line
+  * #897, ActiveSdb alpha release
+
+== 1.7.1
+
+  Do not autoload right_sdb with the rest of the modules; it requires uuidtools.
+    We want the user to explicly request ActiveSdb, at least as long as it is
+    alpha/beta.
+
+  Fix escaping problem in SqsGen2Interface: POST bodies did not properly escape the '&' character
+
+== 1.7.2
+
+  Release Notes:
+  
+  RightAws includes some new features, including:
+  - Support in RightAws::S3 and RightAws::S3Interface for S3 key copy, move, and rename
+  - Support for signature version 0 request authentication to EC2, SQS, and SDB
+  - Enhanced S3 object meta-header read and update
+  - Interoperability with clouds running Eucalyptus (http://eucalyptus.cs.ucsb.edu)
+    [ Contributed by the Eucalyptus group ]
+  - Support for c1.medium and c1.xlarge instance types
+
+  Bug fixes include:
+  - Corrected the failure, under certain conditions, of retries of streaming PUTs to S3.
+    We now reset the seek pointer of the streaming IO object to its initial position.
+  - Removal of an accidental dependency on ActiveSupport in RightAws::S3Interface.get_link().
+  - Monkey-patch of the Ruby File class on Windows platforms to correct a problem in lstat.
+    The lstat bug was causing failure of very large file uploads on Windows [ Contributed by Benjamin Allfree ] 
+  - Fixed parsing of the ETag field for S3 objects
+
+== 1.7.3
+
+  Release Notes:
+
+  - Removed the 1.7.2 monkey-patch of the Ruby File class on Windows.  This patch broke Rails 2.0.
+    The patch is now included in the README for anyone to use at their own risk.
+ 
+== 1.8.0
+
+  Release Notes:
+
+  This release adds major new features to RightAws to support Amazon's new
+  Elastic Block Store (EBS).  Via the RightAws::Ec2 module, users can create
+  and delete EBS volumes, attach and detach them from instances, snapshot
+  volumes, and list the available volumes and snapshots.
+
+  Bug fixes include correction of RightAws::S3 copy's failure to url-encode
+the source key.
+ 
+== 1.8.1
+
+  Release Notes:
+
+  RightScale::SdbInterface & ::ActiveSdb have several enhancements, including:
+  - RightAws::SdbInterface#last_query_expression added for debug puposes
+  - RightAws::ActiveSdb::Base#query :order and :auto_load options added to support query
+    result sorting and attributes auto loading
+  - RightAws::ActiveSdb::Base#find_all_by_ and find_by_ helpers improved to support
+    :order, :auto_load, :limit and :next_token options
+  - RightAws::SdbInterface#delete_attributes bug fixed 
+  - SdbInterface allows specification of a string value to use for
+    representing Ruby nil in SDB.
+  - Sdb tests fixed and improved
+
+  The ::S3 interface now has support for S3's server access logging.
+  Amazon considers server access logging to be a beta or provisional feature.
+
+=== 1.9.0
+
+  Release Notes:
+  - RightAws::Ec2 now supports Windows instances. Added:
+   - Ec2::get_initial_password
+   - Ec2::bundle_instance
+   - Ec2::describe_bundle_tasks
+   - Ec::cancel_bundle_task
+
+  - Full Amazon CloudFront support added with RightAws::AcfInterface
+  - Bug fixes to S3Interface::store_object_and_verify and 
+    S3Interface::retrieve_object_and_verify (thanks to numerous user reports)
+  - Updates to caching for Ec2::describe_images_by methods
+  - Ec2 now has Ec2::last_request_id
+
+=== 1.10.0
+
+  Release Notes:
+
+  The big new features are SDB's SQL-like query and query_with_attributes
+  support as well as signature v2 support for all services.  There are also
+  numerous bug fixes, many of them reported and patched by users and
+  customers.
+
+  - AwsBase: signature v2 support added
+  - AwsBase: fix the regex matching for xmlpath which didn't work in Ruby
+    1.8.7 (thanks to a customer report)
+  - Ec2: describe_availability_zones improved to support regions
+  - Ec2: Disabled retries when EC2 returns "ServiceUnavailable: Request limit
+    exceeded"
+  - Ec2: Use POST for large queries; this avoids truncation of large user data
+    when launching instances (thanks to Bob for the report)
+  - CloudFront: docs fixes
+  - SDB: added: SQL-like query, select and query_with_attributes support
+  - SDB: fixed no method error when searching for id that doesn't exist
+    (thanks to multiple users for reporting this)
+  - S3: Fixed overzealous URL-encoding when generating URLs for S3 keys
+    (thanks to a bug report on the RubyForge forum)
+  

data/Manifest.txt

@@ -0,0 +1,34 @@
+History.txt
+Manifest.txt
+README.txt
+Rakefile
+lib/awsbase/benchmark_fix.rb
+lib/awsbase/right_awsbase.rb
+lib/awsbase/support.rb
+lib/ec2/right_ec2.rb
+lib/right_aws.rb
+lib/s3/right_s3.rb
+lib/s3/right_s3_interface.rb
+lib/sdb/active_sdb.rb
+lib/sdb/right_sdb_interface.rb
+lib/sqs/right_sqs.rb
+lib/sqs/right_sqs_gen2.rb
+lib/sqs/right_sqs_gen2_interface.rb
+lib/sqs/right_sqs_interface.rb
+lib/acf/right_acf_interface.rb
+test/ec2/test_helper.rb
+test/ec2/test_right_ec2.rb
+test/http_connection.rb
+test/s3/test_helper.rb
+test/s3/test_right_s3.rb
+test/s3/test_right_s3_stubbed.rb
+test/sdb/test_active_sdb.rb
+test/sdb/test_helper.rb
+test/sdb/test_right_sdb.rb
+test/sqs/test_helper.rb
+test/sqs/test_right_sqs.rb
+test/sqs/test_right_sqs_gen2.rb
+test/test_credentials.rb
+test/ts_right_aws.rb
+test/acf/test_helper.rb
+test/acf/test_right_acf.rb

data/README.txt

@@ -0,0 +1,166 @@
+= RightScale Amazon Web Services Ruby Gems
+
+Published by RightScale, Inc. under the MIT License.
+For information about RightScale, see http://www.rightscale.com
+
+== DESCRIPTION:
+
+The RightScale AWS gems have been designed to provide a robust, fast, and secure interface to Amazon EC2, EBS, S3, SQS, SDB, and CloudFront. 
+These gems have been used in production by RightScale since late 2006 and are being maintained to track enhancements made by Amazon. 
+The RightScale AWS gems comprise:
+
+- RightAws::Ec2 -- interface to Amazon EC2 (Elastic Compute Cloud) and the
+  associated EBS (Elastic Block Store)
+- RightAws::S3 and RightAws::S3Interface -- interface to Amazon S3 (Simple Storage Service)
+- RightAws::Sqs and RightAws::SqsInterface -- interface to first-generation Amazon SQS (Simple Queue Service) (API version 2007-05-01)
+- RightAws::SqsGen2 and RightAws::SqsGen2Interface -- interface to second-generation Amazon SQS (Simple Queue Service) (API version 2008-01-01)
+- RightAws::SdbInterface and RightAws::ActiveSdb -- interface to Amazon SDB (SimpleDB)
+- RightAws::AcfInterface -- interface to Amazon CloudFront, a content distribution service
+
+== FEATURES:
+
+- Full programmmatic access to EC2, EBS, S3, SQS, SDB, and CloudFront.
+- Complete error handling: all operations check for errors and report complete
+  error information by raising an AwsError.
+- Persistent HTTP connections with robust network-level retry layer using
+  RightHttpConnection).  This includes socket timeouts and retries.
+- Robust HTTP-level retry layer.  Certain (user-adjustable) HTTP errors returned
+  by Amazon's services are classified as temporary errors.
+  These errors are automaticallly retried using exponentially increasing intervals.
+  The number of retries is user-configurable.
+- Fast REXML-based parsing of responses (as fast as a pure Ruby solution allows).
+- Uses libxml (if available) for faster response parsing. 
+- Support for large S3 list operations.  Buckets and key subfolders containing
+  many (> 1000) keys are listed in entirety.  Operations based on list (like
+  bucket clear) work on arbitrary numbers of keys.
+- Support for streaming GETs from S3, and streaming PUTs to S3 if the data source is a file.
+- Support for single-threaded usage, multithreaded usage, as well as usage with multiple
+  AWS accounts.
+- Support for both first- and second-generation SQS (API versions 2007-05-01
+  and 2008-01-01).  These versions of SQS are not compatible.
+- Support for signature versions 0, 1 and 2 on all services.
+- Interoperability with any cloud running Eucalyptus (http://eucalyptus.cs.ucsb.edu)
+- Test suite (requires AWS account to do "live" testing).
+
+== THREADING:
+
+All RightScale AWS interfaces offer two threading options:
+1. Use a single persistent HTTP connection per process.
+2. Use a persistent HTTP connection per Ruby thread.
+ 
+Either way, it doesn't matter how many (for example) RightAws::S3 objects you create,
+they all use the same per-program or per-thread
+connection. The purpose of sharing the connection is to keep a single
+persistent HTTP connection open to avoid paying connection
+overhead on every request. However, if you have multiple concurrent
+threads, you may want or need an HTTP connection per thread to enable
+concurrent requests to AWS. The way this plays out in practice is:
+1. If you have a non-multithreaded Ruby program, use the non-multithreaded setting.
+2. If you have a multi-threaded Ruby program, use the multithreaded setting to enable
+   concurrent requests to S3 (or SQS, or SDB, or EC2).
+3. For running under Mongrel/Rails, use the non-multithreaded setting even though
+   mongrel is multithreaded.  This is because only one Rails handler is invoked at
+   time (i.e. it acts like a single-threaded program)
+
+Note that due to limitations in the I/O of the Ruby interpreter you
+may not get the degree of parallelism you may expect with the multi-threaded setting.
+
+By default, EC2/S3/SQS/SDB/ACF interface instances are created in single-threaded mode.  Set
+"params[:multi_thread]" to "true" in the initialization arguments to use
+multithreaded mode.
+
+== GETTING STARTED:
+
+* For EC2 read RightAws::Ec2 and consult the Amazon EC2 API documentation at
+  http://developer.amazonwebservices.com/connect/kbcategory.jspa?categoryID=87
+* For S3 read RightAws::S3 and consult the Amazon S3 API documentation at
+  http://developer.amazonwebservices.com/connect/kbcategory.jspa?categoryID=48
+* For first generation SQS read RightAws::Sqs and consult the Amazon SQS API documentation at
+  http://developer.amazonwebservices.com/connect/kbcategory.jspa?categoryID=31
+* For second generation SQS read RightAws::SqsGen2, RightAws::SqsGen2Interface, and consult the Amazon SQS API documentation at
+  http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1214&categoryID=31 
+
+  Amazon's Migration Guide for moving from first to second generation SQS is
+  avalable at:
+  http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1148
+* For SDB read RightAws::SdbInterface, RightAws::ActiveSdb, and consult the Amazon SDB API documentation at
+  http://developer.amazonwebservices.com/connect/kbcategory.jspa?categoryID=141
+* For CloudFront (ACF) read RightAws::AcfInterface and consult the Amazon CloudFront API documentation at 
+  http://developer.amazonwebservices.com/connect/kbcategory.jspa?categoryID=213
+
+== KNOWN ISSUES:
+
+- 7/08: A user has reported that uploads of large files on Windows may be broken on some
+  Win platforms due to a buggy File.lstat.size.  Use the following monkey-patch at your own risk, 
+  as it has been proven to break Rails 2.0 on Windows:
+
+    require 'win32/file'
+    class File
+      def lstat
+        self.stat
+      end
+    end
+
+
+- Attempting to use the Gibberish plugin (used by the Beast forum app) 
+  will break right_aws as well as lots of other code.  Gibberish
+  changes the semantics of core Ruby (specifically, the String class) and thus presents a reliability
+  problem for most Ruby programs.
+
+- 2/11/08: If you use RightAws in conjunction with attachment_fu, the
+  right_aws gem must be included (using the require statement) AFTER
+  attachment_fu.  If right_aws is loaded before attachment_fu, you'll
+  encounter errors similar to:
+
+  s3.amazonaws.com temporarily unavailable: (wrong number of arguments (5 for 4))
+
+  or
+
+  'incompatible Net::HTTP monkey-patch'
+
+  This is due to a conflict between the right_http_connection gem and another
+  gem required by attachment_fu.  It may be possible to require right_aws (and
+  thus right_http_connection) in the .after_initialize method of the config object in
+  environment.rb (check the docs for Rails::Configuration.after_initialize).
+
+- 8/07: Amazon has changed the semantics of the SQS service.  A
+  new queue may not be created within 60 seconds of the destruction of any
+  older queue with the same name.  Certain methods of RightAws::Sqs and
+  RightAws::SqsInterface will fail with the message:
+  "AWS.SimpleQueueService.QueueDeletedRecently: You must wait 60 seconds after deleting a queue before you can create another with the same name."
+  
+== REQUIREMENTS:
+
+RightAws requires REXML and the right_http_connection gem.
+If libxml and its Ruby bindings (distributed in the libxml-ruby gem) are
+present, RightAws can be configured to use them:
+  RightAws::RightAWSParser.xml_lib = 'libxml'
+Any error with the libxml installation will result in RightAws failing-safe to
+REXML parsing.
+
+== INSTALL:
+
+sudo gem install right_aws
+
+== LICENSE:
+
+Copyright (c) 2007-2009 RightScale, Inc. 
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+'Software'), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/Rakefile

@@ -0,0 +1,112 @@
+# -*- ruby -*-
+
+require 'rubygems'
+require 'hoe'
+require "rake/testtask"
+require 'rcov/rcovtask'
+$: << File.dirname(__FILE__)
+require 'lib/right_aws.rb'
+
+testglobs =     ["test/ts_right_aws.rb"]
+
+
+# Suppress Hoe's self-inclusion as a dependency for our Gem. This also keeps
+# Rake & rubyforge out of the dependency list. Users must manually install
+# these gems to run tests, etc.
+# TRB 2/24/09: also do this for the extra_dev_deps array present in newer hoes.
+# Older versions of RubyGems will try to install developer-dependencies as
+# required runtime dependencies.  It would be great to take advantage of the extra dev deps, but not
+# at the cost of requiring many additional dependencies if the user is running an older RubyGems.
+class Hoe
+  def extra_deps
+    @extra_deps.reject do |x|
+      Array(x).first == 'hoe'
+    end
+  end
+  def extra_dev_deps
+    @extra_dev_deps.reject do |x|
+      Array(x).first == 'hoe'
+    end
+  end
+end
+
+Hoe.new('dmarkov-right_aws', RightAws::VERSION::STRING) do |p|
+  p.rubyforge_name = 'rightscale'
+  p.author = 'RightScale, Inc.'
+  p.email = 'rubygems@rightscale.com'
+  p.summary = 'Interface classes for the Amazon EC2/EBS, SQS, S3, SDB, and ACF Web Services'
+  p.description = p.paragraphs_of('README.txt', 2..5).join("\n\n")
+  p.url = p.paragraphs_of('README.txt', 0).first.split(/\n/)[1..-1]
+  p.changes = p.paragraphs_of('History.txt', 0..1).join("\n\n")
+  p.remote_rdoc_dir = "/right_aws_gem_doc"
+  p.extra_deps = [['right_http_connection','>= 1.2.4']]
+  p.test_globs = testglobs 
+end
+
+desc "Analyze code coverage of the unit tests."
+Rcov::RcovTask.new do |t|
+  t.test_files = FileList[testglobs]
+  #t.verbose = true     # uncomment to see the executed command
+end
+ 
+desc "Test just the SQS interface"
+task :testsqs do
+  require 'test/test_credentials'
+  require 'test/http_connection'
+  TestCredentials.get_credentials
+  require 'test/sqs/test_right_sqs.rb'
+end
+
+desc "Test just the second generation SQS interface"
+task :testsqs2 do
+  require 'test/test_credentials'
+  require 'test/http_connection'
+  TestCredentials.get_credentials
+  require 'test/sqs/test_right_sqs_gen2.rb'
+end
+
+desc "Test just the S3 interface"
+task :tests3 do
+  require 'test/test_credentials'
+  require 'test/http_connection'
+  TestCredentials.get_credentials
+  require 'test/s3/test_right_s3.rb'
+end
+
+desc "Test just the S3 interface using local stubs"
+task :tests3local do
+  require 'test/test_credentials'
+  require 'test/http_connection'
+  TestCredentials.get_credentials
+  require 'test/s3/test_right_s3_stubbed.rb'
+end
+
+desc "Test just the EC2 interface"
+task :testec2 do
+  require 'test/test_credentials'
+  TestCredentials.get_credentials
+  require 'test/ec2/test_right_ec2.rb'
+end
+
+desc "Test just the SDB interface"
+task :testsdb do
+  require 'test/test_credentials'
+  TestCredentials.get_credentials
+  require 'test/sdb/test_right_sdb.rb'
+end
+
+desc "Test active SDB interface"
+task :testactivesdb do
+  require 'test/test_credentials'
+  TestCredentials.get_credentials
+  require 'test/sdb/test_active_sdb.rb'
+end
+
+desc "Test CloudFront interface"
+task :testacf do
+  require 'test/test_credentials'
+  TestCredentials.get_credentials
+  require 'test/acf/test_right_acf.rb'
+end
+
+# vim: syntax=Ruby