awsraw 0.1.9 → 1.0.0.alpha.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: c2162640827a7ebe53d58fd6f53c8ff5c515f80f
-  data.tar.gz: 9b0725c26c0c27465a3a606094ea7e2beafa5def
+  metadata.gz: 74ab136d9a8688640905f2ffd967f828dc960f42
+  data.tar.gz: d961172c156fe8a271b94d33665597cea23e249b
 SHA512:
-  metadata.gz: d8b3bf71d146932f0def424d13105286833ee2635387d6fb4ae968a02a902e0b050773e404c77ebbf5a9a54e2a8202cfab5b8edf4934c5614ab793b7c496f236
-  data.tar.gz: 68fb736100f5693ced2f453e311fd7d8ca96edab5c124364a45be46db6969319bb3937c7fa9015db55450dafdfe9558654dae6d2019857e907310eac85d438b8
+  metadata.gz: 2512ee73dde56b39b937258e0432eacd3b12a2056905ae28b921df5afb758d7d98cafe8225f3d96cefbcb9daf2833c42037724e8e9cc04ff57024f64b1c22d6d
+  data.tar.gz: c39c859a8b5516efaf9fd33dd0afc234c23efa70d79d802571b0baf68aebc81af417afcaae06a5886b830e46be35a9f9a6ea02e76f37f0dd92fe9882f559dab3

data/README.md

@@ -1,76 +1,148 @@
-# AWSRaw [![Build Status](https://travis-ci.org/envato/awsraw.svg?branch=0.1-maintenance)](https://travis-ci.org/envato/awsraw)
+# AWSRaw
 
 A client for [Amazon Web Services](http://www.amazonaws.com/) in the style of
-[FlickRaw](http://hanklords.github.com/flickraw/)
+[FlickRaw](http://hanklords.github.com/flickraw/).
 
 ## Background
 
-AWSRaw helps you make authenticated requests to AWS's various services. It
-doesn't provide any higher-level concepts like, for example, "delete this
-file from S3". Instead, you should understand S3's http API and know that
-sending a `DELETE` request to the bucket/key URL will result in the file
-being deleted.
+AWSRaw has a simple goal: to let you follow the [AWS API
+docs](http://aws.amazon.com/documentation/), and translate that into Ruby code
+with the minimum of fuss.
 
-While these higher-level concepts can be useful (see, e.g.,
-[fog](https://github.com/fog/fog)), they can also get in the way. Being
-able to use a new AWS feature by simply following the AWS docs' examples
-directly is very nice, instead of having to dig deep into a higher-level
-library to figure out how they've mapped that new feature into their
-terminology and API.
+This is the opposite of [fog](http://fog.io). AWSRaw tries to add as little
+abstraction as possible on top of the AWS REST API.
 
-## Configuration
+You use a regular HTTP library to make requests, and AWSRaw provides useful
+additions like request signing.
 
-If you need to override the AWS hostname for development/testing purposes, you can do so as follows:
+
+## Examples
+
+### Credentials
+
+For all the examples below, you'll need to set up your credentials like this:
 
 ```ruby
-require 'awsraw/s3/client'
-
-# Assuming we have a fake S3 service listening on `fake.s3.dev`
-AWSRaw::S3.configure do |config|
-  config.host = 'fake.s3.dev'
-  config.regional_hosts = {
-    'ap-southeast-2' => 'fake.s3.dev'
-  }
-end
+credentials = AWSRaw::Credentials.new(
+  :access_key_id     => "...",
+  :secret_access_key => "..."
+)
 ```
 
-## Usage
-
 ### S3
 
-Standard requests:
+Set up your Faraday connection something like this:
 
 ```ruby
-require 'awsraw/s3/client'
+connection = Faraday.new("http://s3.amazonaws.com") do |faraday|
+  faraday.use      AWSRaw::S3::FaradayMiddleware, credentials
+  faraday.response :logger
+  faraday.adapter  Faraday.default_adapter
+end
+```
 
-s3 = AWSRaw::S3::Client.new(
-       ENV['AWS_ACCESS_KEY_ID'],
-       ENV['AWS_SECRET_ACCESS_KEY'])
+A simple GET request:
 
-file = File.open("reaction.gif", "rb")
+```ruby
+response = connection.get("/mah-sekret-buckit/reaction.gif")
+```
 
-s3.request(:method  => "PUT",
-           :bucket  => "mah-sekret-buckit",
-           :key     => "/reaction.gif",
-           :content => file,
-           :headers => { "Content-Type" => "image/gif" })
+A PUT request:
 
-f.close
+```ruby
+connection.put do |request|
+  request.url "/mah-sekret-buckit/reaction.gif"
+  request.headers["Content-Type"] = "image/gif"
+  request.body = File.new("reaction.gif")
+end
 ```
 
-Signed query-string requests, to allow authorized clients to get protected
-resources:
+See the [AWS S3 REST API docs](http://docs.aws.amazon.com/AmazonS3/latest/API/APIRest.html)
+for all the requests you can make.
+
+
+#### On request bodies
+
+If your request has a body and you don't provide a Content-MD5 header for it,
+AWSRaw will try to calculate one. (The S3 API requires the Content-MD5 header
+for correct request signing.)
+
+It can handle the body behaving as either a String or a File. If you want to do
+something different with the body, you'll need to set the Content-MD5 header
+yourself.
+
+You must also provide a Content-Type header for your request if there's a
+request body. AWSRaw will raise an exception if you don't.
+
+
+#### Signing query strings
+
+If you need a signed URI with an expiry date, this is how to do it. See the
+[AWS docs on the
+subject](http://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAuthentication.html#RESTAuthenticationQueryStringAuth).
+
 
 ```ruby
-require 'awsraw/s3/query_string_signer'
+signer = AWSRaw::S3::QueryStringSigner.new(credentials)
+
+uri = signer.sign(
+  "https://s3.amazonaws.com/mah-sekret-buckit/reaction.gif",
+  Time.now + 600 # The URI will expire in 10 minutes.
+)
+```
+
+
+#### HTML Form Uploads
 
-signer = AWSRaw::S3::QueryStringSigner.new(
-           ENV['AWS_ACCESS_KEY_ID'],
-           ENV['AWS_SECRET_ACCESS_KEY'])
+You can use AWSRaw to generate signatures for browser-based uploads. See the
+[AWS docs on the
+topic](http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingHTTPPOST.html).
 
-url = "http://s3.amazonaws.com/mah-sekret-bucket/reaction.gif"
-expiry = Time.now.utc + 60 # 1 minute from now
-temporary_url = signer.sign_with_query_string(url, expiry.to_i)
-puts temporary_url
-  # => "http://s3.amazonaws.com/mah-sekret-bucket/reaction.gif?Signature=..."
+```ruby
+policy = [
+  { "bucket" => "mah-secret-buckit" }
+]
+
+policy_json = JSON.generate(policy)
+
+http_post_variables = {
+  "AWSAccessKeyID" => credentials.access_key_id,
+  "key"            => "reaction.gif",
+  "policy"         => AWSRaw::S3::Signature.encode_form_policy(policy_json),
+  "signature"      => AWSRaw::S3::Signature.form_signature(policy_json, credentials)
+}
 ```
+
+Then get your browser to do an XHR request using the http_post_variables, and
+Bob's your aunty.
+
+
+## Status
+
+This is still a bit experimental, and is missing some key features, but what's
+there is solid and well tested.
+
+Right now AWSRaw only has direct support for
+[Faraday](https://github.com/lostisland/faraday), but you could still use it
+with other client libraries with a bit of work.
+
+So far we've only built S3 support. We'd love to see pull requests for other
+AWS services.
+
+
+## Contributing
+
+1. Fork it
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Add some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create new Pull Request
+
+
+## To Do
+
+- Add smart handling of errors
+    - Identify cases where string-to-sign doesn't match, and display something helpful
+    - Raise exceptions for errors?
+- Add easy ways to nicely format XML responses
+

data/Rakefile

@@ -1,7 +1 @@
 require "bundler/gem_tasks"
-require "rspec/core/rake_task"
-
-desc "Run tests"
-RSpec::Core::RakeTask.new(:spec)
-
-task :default => [:spec]

data/awsraw.gemspec

@@ -4,9 +4,9 @@ require "awsraw/version"
 
 Gem::Specification.new do |s|
   s.name        = "awsraw"
-  s.version     = Awsraw::VERSION
-  s.authors     = ["Pete Yandell", "David Goodlad", "Jack 'chendo' Chen", "Warren Seen"]
-  s.email       = ["pete@notahat.com", "david@goodlad.net", "gems.awsraw@chen.do", "warren@warrenseen.com"]
+  s.version     = AWSRaw::VERSION
+  s.authors     = ["Pete Yandell", "David Goodlad", "Jack 'chendo' Chen"]
+  s.email       = ["pete@notahat.com", "david@goodlad.net", "gems.awsraw@chen.do"]
   s.license     = 'MIT'
   s.homepage    = "http://github.com/envato/awsraw"
   s.summary     = %q{Minimal AWS client}
@@ -21,6 +21,7 @@ Gem::Specification.new do |s|
 
   # specify any dependencies here; for example:
   s.add_development_dependency "rake"
-  s.add_development_dependency "rspec"
-  # s.add_runtime_dependency "rest-client"
+  s.add_development_dependency "rspec", "~> 2.14"
+  s.add_runtime_dependency "finer_struct", "~> 0.0.5"
+  s.add_runtime_dependency "faraday", "~> 0.8.8"
 end

data/lib/awsraw.rb

@@ -1,5 +1,10 @@
+require "awsraw/error"
 require "awsraw/version"
+require "awsraw/s3/canonicalized_resource"
+require "awsraw/s3/client"
+require "awsraw/s3/content_md5_header"
+require "awsraw/s3/faraday_middleware"
+require "awsraw/s3/query_string_signer"
+require "awsraw/s3/signature"
+require "awsraw/s3/string_to_sign"
 
-module AWSRaw
-
-end

data/lib/awsraw/credentials.rb

@@ -0,0 +1,7 @@
+require 'finer_struct'
+
+module AWSRaw
+  class Credentials < FinerStruct::Immutable(:access_key_id, :secret_access_key)
+  end
+end
+

data/lib/awsraw/error.rb

@@ -0,0 +1,4 @@
+module AWSRaw
+  class Error < RuntimeError
+  end
+end

data/lib/awsraw/s3/canonicalized_resource.rb

@@ -0,0 +1,50 @@
+require 'uri'
+
+module AWSRaw
+  module S3
+
+    module CanonicalizedResource
+
+      def self.canonicalized_resource(uri)
+        uri = URI(uri)
+        bucket = bucket_from_hostname(uri.hostname)
+
+        [bucket && "/#{bucket}", uri.path, canonicalized_subresources(uri.query)].join
+      end
+
+      # Extract the bucket name from the hostname for virtual-host-style and
+      # cname-style S3 requests. Returns nil for path-style requests.
+      #
+      # See: http://docs.aws.amazon.com/AmazonS3/latest/dev/VirtualHosting.html
+      def self.bucket_from_hostname(hostname)
+        if hostname =~ %r{s3[-\w\d]*\.amazonaws\.com$}
+          components = hostname.split(".")
+          if components.length > 3
+            components[0..-4].join(".")
+          else
+            nil
+          end
+        else
+          hostname
+        end
+      end
+
+      VALID_SUBRESOURCES = %w{acl lifecycle location logging notification partNumber policy requestPayment torrent uploadId uploads versionId versioning versions website}
+
+      # Generates the canonicalized subresources for a URI, as per:
+      #
+      # http://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAuthentication.html
+      #
+      # Note: This is incomplete, in that it doesn't handle header values
+      # that are overridden by parameters, nor does it handle the "delete"
+      # parameter for multi-object Delete requests.
+      def self.canonicalized_subresources(query)
+        query ||= ""
+        subresources = query.split("&") & VALID_SUBRESOURCES
+        "?#{subresources.sort.join('&')}" unless subresources.empty?
+      end
+
+    end
+
+  end
+end

data/lib/awsraw/s3/client.rb

@@ -1,35 +1,42 @@
-require 'net/http'
-require 'awsraw/s3'
-require 'awsraw/s3/request'
-require 'awsraw/s3/http_request_builder'
-require 'awsraw/s3/response'
-require 'awsraw/s3/signer'
-require 'awsraw/s3/md5_digester'
+require 'uri'
+require 'faraday'
+require 'awsraw/credentials'
+require 'awsraw/s3/faraday_middleware'
+
 module AWSRaw
   module S3
 
-    class ConnectionError < StandardError; end
-
-    # A client for the AWS S3 rest API.
-    #
-    # http://docs.amazonwebservices.com/AmazonS3/latest/API/APIRest.html
+    # Legacy client, to support AWSRaw pre-1.0 requests. You shouldn't be using
+    # this anymore.
     class Client
 
       def initialize(access_key_id, secret_access_key)
-        @access_key_id     = access_key_id
-        @secret_access_key = secret_access_key
+        @credentials = AWSRaw::Credentials.new(
+          :access_key_id     => access_key_id,
+          :secret_access_key => secret_access_key
+        )
       end
 
       def request(params = {})
-        request = Request.new(params, signer)
-
-        http_request = HTTPRequestBuilder.new(request).build
+        host = params[:region] ? "s3-#{params[:region]}.amazonaws.com" : "s3.amazonaws.com"
+        path = URI.escape("/#{params[:bucket]}#{params[:key]}")
+        url = URI::HTTP.build(
+          :host  => host,
+          :path  => path,
+          :query => params[:query]
+        )
 
-        http_response = Net::HTTP.start(request.uri.host, request.uri.port) do |http|
-          http.request(http_request)
+        faraday_response = connection.send(params[:method].downcase) do |request|
+          request.url(url)
+          request.headers = params[:headers] || {}
+          request.body    = params[:content]
         end
 
-        construct_response(http_response)
+        Response.new(
+          :code    => faraday_response.status,
+          :headers => faraday_response.headers,
+          :content => faraday_response.body
+        )
       end
 
       def request!(params = {})
@@ -39,18 +46,33 @@ module AWSRaw
 
     private
 
-      def construct_response(http_response)
-        Response.new(
-          :code    => http_response.code,
-          :headers => http_response.to_hash,
-          :content => http_response.body
-        )
+      def connection
+        @connection ||= Faraday.new do |faraday|
+          faraday.use     AWSRaw::S3::FaradayMiddleware, @credentials
+          faraday.adapter Faraday.default_adapter
+        end
+      end
+
+    end
+
+    class Response
+      def initialize(params = {})
+        @code    = params[:code]
+        @headers = params[:headers]
+        @content = params[:content]
       end
 
-      def signer
-        @signer ||= Signer.new(@access_key_id, @secret_access_key)
+      attr_accessor :code
+      attr_accessor :headers
+      attr_accessor :content
+
+      def success?
+        code =~ /^2\d\d$/
       end
 
+      def failure?
+        !success?
+      end
     end
 
   end

data/lib/awsraw/s3/content_md5_header.rb

@@ -0,0 +1,34 @@
+require 'digest/md5'
+
+module AWSRaw
+  module S3
+    module ContentMD5Header
+
+      def self.generate_content_md5(body)
+        return nil if body.nil?
+
+        digest = Digest::MD5.new
+        if body.respond_to?(:read)
+          read_file_into_digest(digest, body)
+        else
+          digest << body
+        end
+
+        digest.base64digest
+      end
+
+    private
+
+      # This mimics the behaviour of Ruby's Digest::Instance#file method.
+      # Unfortunately that takes a filename not a file, so we can't use it.
+      def self.read_file_into_digest(digest, file)
+        buffer = ""
+        while file.read(16384, buffer)
+          digest << buffer
+        end
+        file.rewind # ...so the HTTP client can read the body for sending.
+      end
+
+    end
+  end
+end