px-service-client 1.2.0 → 1.2.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 260462f7c8c782f9a5a1b7a0996431e426f01f39
-  data.tar.gz: 31ffc0fff0138c3d8519d523da451b23e765de0c
+  metadata.gz: e91b9e1a396db568b3b30155336363538b80fff6
+  data.tar.gz: 72792c99dc994aeed39b1e0a0b586c486bc96e03
 SHA512:
-  metadata.gz: dc00a6aca11761cc581dc3175a322027e8ee5ac8315933a6c72803665efae80bb5794ba5a25a83c0e831d5b01708f469cd84c79cd3cce8262ab71a5885a4e84f
-  data.tar.gz: 6aefddddf56c6968920130e3aaacddc3bcdf0ef41677d747e651a00cb8afcfe561e4a0341cbcfedf90df0b68cdb0ff848442991b63fc3c9fac66d09a01f87941
+  metadata.gz: 1380aa22511673fc12aef1b7e67d165e002429793d6aefebd6786c9d1f0f65c1db2ec80c9921c6b5382936aacfb3fd437644b7c7a8ceac2ab41c7132aef36bb7
+  data.tar.gz: 45a304051466c4540fbc41dfe21efc454d973e119f4d8637cbce0c947fc213dc3980ea4289fef4f2368d385150afbe01c47d8b91ea19c580aa919c16f74d6604

data/README.md

@@ -23,7 +23,7 @@ Then use it:
 ```ruby
 require 'px-service-client'
 
-class MyClient
+class MyClient < Px::Service::Client::Base
   include Px::Service::Client::Caching
   include Px::Service::Client::CircuitBreaker
 end
@@ -37,8 +37,41 @@ This gem includes several common features used in 500px service client libraries
 
 The features are:
 
+#### Px::Service::Client::Base
+This class provides a basic `make_request(method, url, ...)` method that produces an asynchronous request. The method immediately returns a `RetriableResponseFuture`. It works together with `Multiplexer`(discussed below) and uses [Typhoeus](https://github.com/typhoeus/typhoeus)  as the underlying HTTP client to support asynchronicity. 
+
+**Customized clients usually inherit this class and include other features/mixins, if needed.**  
+
+See the following secion for an example of how to use `make_request` and `Multiplexer`. 
+
+#### Px::Service::Client::Multiplexer
+This class works together with `Px::Service::Client::Base` sub-classes to support request parallel execution. 
+
+Example:
+
+```Ruby
+multi = Px::Service::Client::Multiplexer.new
+
+multi.context do
+	method = :get
+	url = 'http://www.example.com'
+	req = make_request(method, url) # returns a RetriableResponseFuture
+	multi.do(req) # queues the request/future into hydra
+end
+
+multi.run # a blocking call, like hydra.run
+
+```
+`multi.context` encapsulates the block into a [`Fiber`](http://ruby-doc.org/core-2.2.0/Fiber.html) object and immediately runs (or `resume`, in Fiber's term) that fiber until the block explicitly gives up control. The method returns `multi` itself. 
+
+`multi.do(request_or_future,retries)` queues the request into `hydra`. It always returns a `RetriableResponseFuture`. A  [`Typhoeus::Request`](https://github.com/typhoeus/typhoeus) will be converted into a `RetriableResponseFuture ` in this call. 
+
+Finally, `multi.run` starts `hydra` to execute the requests in parallel. The request is made as soon as the multiplexer is started. You get the results of the request by evaluating the value of the `RetriableResponseFuture`.
+
 #### Px::Service::Client::Caching
 
+Provides client-side response caching of service requests.  
+
 ```ruby
 include Px::Service::Client::Caching
 
@@ -52,36 +85,33 @@ caching do |config|
   config.cache_client =  Dalli::Client.new(...)
   config.cache_logger = Logger.new(STDOUT) # or Rails.logger, for example
 end
+
+# An example of a cached request
+result = cache_request(url, :last_resort, refresh_probability: 1) do
+	req = make_request(method, url)
+end
 ```
 
-Provides client-side response caching of service requests.  Responses are cached in memcached (using the provided cache client) in either a *last-resort* or *first-resort* manner.
+`cache_request` expects a block that does the `make_request` and returns a `RetriableResponseFuture`. The block takes no argument. If neither the cache nor the response has the data, the exception `ServiceError` will be re-raised. 
+
+Responses are cached in memcached (using the provided cache client) in either a *last-resort* or *first-resort* manner.
 
 *last-resort* means that the cached value is only used when the service client request fails (with a
-`ServiceError`).  When using last-resort caching, a `refresh_probability` can be provided that causes the cached value
-to be refreshed probabilistically (rather than on every request).
+`ServiceError`). If the service client request succeeds, there is a chance that the cache value may get refreshed. The `refresh_probability` is provided to let the cached value
+be refreshed probabilistically (rather than on every request). 
 
-*first-resort* means that the cached value is always used, if present.  Requests to the service are only made
-when the cached value is close to expiry.
+If the service client request fails and there is a `ServiceError`, `cache_logger` will record the exception message, and attempt to read the existing cache value. 
 
-An example of a cached request:
+*first-resort* means that the cached value is always used, if present.  If the cached value is present but expired, the it sends the service client request and, if the request succeeds, it refreshes the cached value expiry. If the request fails, it uses the expired cached value, but the value remain expired. A retry may be needed. 
 
-```ruby
-req = subject.make_request(method, url)
-result = subject.cache_request(url) do
-  resp = nil
-  multi.context do
-    resp = multi.do(req)
-  end.run
-
-  resp
-end
-```
 
-`cache_request` expects a block that returns a `RetriableResponseFuture`. It then returns a `Typhoeus::Response`.
 
 #### Px::Service::Client::CircuitBreaker
+This mixin overrides `Px::Service::Client::Base#make_request` method and implements the circuit breaker pattern. 
 
 ```ruby
+include Px::Service::Client::CircuitBreaker
+
 # Optional
 circuit_handler do |handler|
  handler.logger = Logger.new(STDOUT)
@@ -90,25 +120,67 @@ circuit_handler do |handler|
  handler.invocation_timeout = 10
  handler.excluded_exceptions += [NotConsideredFailureException]
 end
+
+# An example of a make a request with circuit breaker
+req = make_request(method, url) # overrides Px::Service::Client::Base
 ```
 
-Adds a circuit breaker to the client.  The circuit will open on any exception from the wrapped method, or if the request runs for longer than the `invocation_timeout`.
+Adds a circuit breaker to the client.  `make_request` always returns `RetriableResponseFuture`
+
+The circuit will open on any exception from the wrapped method, or if the request runs for longer than the `invocation_timeout`.
 
-Note that `Px::Service::ServiceRequestError` exceptions do NOT trip the breaker, as these exceptions indicate an error on the caller's part (e.g. an HTTP 4xx error).
+If the circuit is open, any future request will be get an error message wrapped in `Px::Service::ServiceError`. 
+
+By default, `Px::Service::ServiceRequestError` is excluded by the handler. That is, when the request fails with a `ServiceRequestError` exceptions, the same `ServiceRequestError` will be raised. But it does NOT increase the failure count or trip the breaker, as these exceptions indicate an error on the caller's part (e.g. an HTTP 4xx error).
 
 Every instance of the class that includes the `CircuitBreaker` concern will share the same circuit state.  You should therefore include `Px::Service::Client::CircuitBreaker` in the most-derived class that subclasses
-`Px::Service::Client::Base`
+`Px::Service::Client::Base`.
 
 This module is based on (and uses) the [Circuit Breaker](https://github.com/wsargent/circuit_breaker) gem by Will Sargent.
 
+#### Px::Service::Client::HmacSigning
+Similar to `Px::Service::Client::CircuitBreaker`, this mixin overrides `Px::Service::Client::Base#make_request` method and appends a HMAC signature in the request header. 
+
+To use this mixin: 
+
+```ruby
+class MyClient < Px::Service::Client::Base
+	include Px::Service::Client::HmacSigning
+
+	#optional
+	hmac_signing do |config|
+		config.key = 'mykey'
+		config.keyspan = 300
+	end
+end
+```
+
+Note: `key` and `keyspan` are class variables and shared among instances of the same class. 
+
+The signature is produced from the secret key, a nonce, HTTP method, url, query, body. The nonce is generated from the timestamp. 
+
+To retrieve and verify the signature: 
+
+```ruby
+# Make a request with signed headers
+resp = make_request(method, url, query, headers, body) 
+
+signature = resp.request.options[:headers]["X-Service-Auth"]
+timestamp = resp.request.options[:headers]["Timestamp"]
+
+# Call the class method to regenerate the signature
+expected_signature = MyClient.generate_signature(method, url, query, body, timestamp) 
+
+# assert signature == expected_signature
+```
+
 #### Px::Service::Client::ListResponse
 
 ```ruby
   def get_something(page, page_size)
     response = JSON.parse(http_get("http://some/url?p=#{page}&l=#{page_size}"))
     return Px::Service::Client::ListResponse(page_size, response, "items")
-  end
-
+  end 
 ```
 
 Wraps a deserialized response.  A `ListResponse` implements the Ruby `Enumerable` module, as well

data/lib/px/service/client/base.rb

@@ -1,7 +1,7 @@
 module Px::Service::Client
   class Base
     cattr_accessor :logger
-
+    
     private
 
     def parsed_body(response)
@@ -37,5 +37,6 @@ module Px::Service::Client
 
       RetriableResponseFuture.new(req)
     end
+
   end
 end

data/lib/px/service/client/hmac_signing.rb

@@ -0,0 +1,67 @@
+module Px::Service::Client
+  module HmacSigning
+    extend ActiveSupport::Concern
+    included do
+      alias_method_chain :make_request, :signing
+      
+      cattr_accessor :secret do
+        DEFAULT_SECRET
+      end
+      
+      cattr_accessor  :keyspan do
+        DEFAULT_KEYSPAN
+      end
+      
+    end   
+
+    module ClassMethods      
+      DefaultConfig = Struct.new(:secret, :keyspan) do
+        def initialize
+          self.secret = DEFAULT_SECRET
+          self.keyspan = DEFAULT_KEYSPAN
+        end
+      end
+
+      # initialize the config variables (including secret, keyspan) for hmac siging
+      def hmac_signing(&block)
+        @signing_config = DefaultConfig.new
+        
+        # use default config if no block is given
+        if block_given?
+          yield(@signing_config)        
+          self.secret = @signing_config.secret
+          self.keyspan = @signing_config.keyspan
+        end
+      end
+      
+      ##
+      # Generate a nonce that's used to expire message after keyspan seconds
+      def generate_signature(method, uri, query, body, timestamp)
+        secret = self.secret
+        keyspan = self.keyspan
+        nonce = (timestamp - (timestamp % keyspan)) + keyspan
+        data = "#{method.capitalize},#{uri},#{query},#{body},#{nonce.to_s}"
+        digest = OpenSSL::Digest.new('sha256')
+        digest = OpenSSL::HMAC.digest(digest, secret, data)
+        return Base64.urlsafe_encode64(digest).strip()
+      end
+    end
+    
+    def make_request_with_signing(method, uri, query: nil, headers: nil, body: nil)
+      timestamp = Time.now.to_i
+      signature = self.class.generate_signature(method, uri, query, body, timestamp)
+      
+      headers = {} if headers.nil?
+      headers.merge!("X-Service-Auth" => signature)
+      headers.merge!("Timestamp" => timestamp)
+
+      make_request_without_signing(
+          method,
+          uri,
+          query: query,
+          headers: headers,
+          body: body)
+    end
+
+  end
+end

data/lib/px/service/client/version.rb

@@ -1,7 +1,7 @@
 module Px
   module Service
     module Client
-      VERSION = "1.2.0"
+      VERSION = "1.2.1"
     end
   end
 end

data/lib/px/service/client.rb

@@ -6,6 +6,8 @@ require 'typhoeus'
 module Px
   module Service
     module Client
+      DEFAULT_SECRET = "devsecret"
+      DEFAULT_KEYSPAN = 300
     end
   end
 end
@@ -14,6 +16,7 @@ require "px/service/client/version"
 require "px/service/client/future"
 require "px/service/client/caching"
 require "px/service/client/circuit_breaker"
+require "px/service/client/hmac_signing"
 require "px/service/client/list_response"
 require "px/service/client/base"
 require "px/service/client/multiplexer"

data/spec/px/service/client/hmac_signing_spec.rb

@@ -0,0 +1,45 @@
+require 'spec_helper'
+
+describe Px::Service::Client::HmacSigning do
+  let(:subject_class) {
+    Class.new(Px::Service::Client::Base).tap do |c|
+      # Anonymous classes don't have a name.  Stub out :name so that things work
+      allow(c).to receive(:name).and_return("HmacSigning")
+      c.include(Px::Service::Client::HmacSigning)
+    end
+  }
+  
+  let(:another_class) {
+    Class.new(Px::Service::Client::Base).include(Px::Service::Client::HmacSigning).tap do |c|
+      c.hmac_signing do |signing_config|
+        signing_config.secret = "different secret"
+      end
+    end
+  }
+  
+  subject { subject_class.new }
+  let(:another_object) { another_class.new }
+  
+  describe '#make_request' do
+    context "when the underlying request method succeeds" do
+      let(:url) { 'http://localhost:3000/path' }      
+      let(:resp) { subject.send(:make_request, 'get', url) }
+      let(:headers) { resp.request.options[:headers] }
+
+      it "returns a Future" do
+        expect(resp).to be_a_kind_of(Px::Service::Client::RetriableResponseFuture)
+      end
+
+      it "contains a header with auth signature" do
+        expect(headers).to have_key("X-Service-Auth")
+        expect(headers).to have_key("Timestamp")
+      end
+      
+      let(:resp2) { another_object.send(:make_request, 'get', url) }
+      let(:headers2) { resp2.request.options[:headers] }
+      it "is different from the object of another class with a different key" do
+        expect(headers["X-Service-Auth"]).not_to eq(headers2["X-Service-Auth"])
+      end
+    end
+  end
+end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: px-service-client
 version: !ruby/object:Gem::Version
-  version: 1.2.0
+  version: 1.2.1
 platform: ruby
 authors:
 - Chris Micacchi
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2016-02-26 00:00:00.000000000 Z
+date: 2016-03-31 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: will_paginate
@@ -246,6 +246,7 @@ files:
 - lib/px/service/client/circuit_breaker.rb
 - lib/px/service/client/circuit_breaker_retriable_response_future.rb
 - lib/px/service/client/future.rb
+- lib/px/service/client/hmac_signing.rb
 - lib/px/service/client/list_response.rb
 - lib/px/service/client/multiplexer.rb
 - lib/px/service/client/retriable_response_future.rb
@@ -256,6 +257,7 @@ files:
 - spec/px/service/client/caching/caching_spec.rb
 - spec/px/service/client/circuit_breaker_spec.rb
 - spec/px/service/client/future_spec.rb
+- spec/px/service/client/hmac_signing_spec.rb
 - spec/px/service/client/list_response_spec.rb
 - spec/px/service/client/multiplexer_spec.rb
 - spec/px/service/client/retriable_response_future_spec.rb
@@ -293,6 +295,7 @@ test_files:
 - spec/px/service/client/caching/caching_spec.rb
 - spec/px/service/client/circuit_breaker_spec.rb
 - spec/px/service/client/future_spec.rb
+- spec/px/service/client/hmac_signing_spec.rb
 - spec/px/service/client/list_response_spec.rb
 - spec/px/service/client/multiplexer_spec.rb
 - spec/px/service/client/retriable_response_future_spec.rb