rest-core 3.1.1 → 3.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 530a9ed8d2e0b7cc07703113eaf17950cc611d12
-  data.tar.gz: ad5eb2846acf18bfacdd88fff480e91bccb4f89e
+  metadata.gz: fbe0fe23ff10ec39cf8a2b367616b1c332bd0d42
+  data.tar.gz: 0a7132a6620ecab1c35fd69fdff24733161aafad
 SHA512:
-  metadata.gz: 4e43365d45547c82c1093963a677c34c2f38ab7c637e80c1219462b7a75313aa30ed8ce817ec94d55d9400a1d993b93bf2e4e203b7a6f4334818086dfa5e8ea9
-  data.tar.gz: a7c962c1876c5b67480708d2dda7e5df1299432cb16aea00afd7173efffc7e22c3596f9d06da640511d2de64df649cc4c927d9bd23e3e1492be502534543f287
+  metadata.gz: d59288134f72ed58c335e743e439479d93de99f6c4278d871733ec845790d65cede5986e276f0a8cd0f0aa39f1cf2ee38325ef468e74cbcb4a08bf3cb01ed2cc
+  data.tar.gz: 73a39d9b95793cc496d05e6c9370e0da5bd7c3ae22374e3ebe4ebc71d3dfd0d2778f885a9b8395fe1c8d38fd143b767df45d57ccbfcb6ae957116a8139ba2c41

data/CHANGES.md

@@ -1,5 +1,64 @@
 # CHANGES
 
+## rest-core 3.2.0 -- 2014-06-27
+
+### Enhancements
+
+* Introduced `RC::JsonResponse::ParseError` which would be a subclass of
+  `RC::Json::ParseError`, and contain the original text before parsing.
+  This should be great for debugging. For example, some servers might
+  return HTML instead JSON, and we would like to read the HTML to learn
+  more why they are doing this.
+
+* Introduced `RC::ParseLink` utility for parsing web links described in
+  [RFC5988](http://tools.ietf.org/html/rfc5988)
+
+* Introduced `RC::Clash` which is a hash wrapper protecting you from getting
+  nils from hashes. This is useful whenever we want to access a value deeply
+  inside a hash. For example: `json['a']['b']['c']['d']` would never fail
+  due to nils. Note that `RC::Clash` is recursive.
+
+* Introduced `RC::Smash` which is a hash wrapper protecting you from getting
+  nils from hashes. This is useful whenever we want to access a value deeply
+  inside a hash. Instead of using multiple layers of subscript operators,
+  we try to use a "path" to specify which value we want. For example:
+  `json['a', 'b', 'c', 'd']` is the same as `json['a']['b']['c']['d']` but
+  with protection from nils in the middle. Note that `RC:Smash` is *not*
+  recursive.
+
+* Introduced `RC::ClashResponse` which would wrap the response inside
+  `RC::Clash`. This is useful along with `RC::JsonResponse`.
+
+* Introduced `RC::SmashResponse` which would wrap the response inside
+  `RC::Smash`. This is useful along with `RC::JsonResponse`.
+
+* Introduced `RC::Client.shutdown` which is essentially the same as
+  `RC::Client.thread_pool.shutdown` and `RC::Client.wait`.
+
+* `RC::ClashResponse` and `RC::SmashResponse` is added into `RC::Universal`
+  with `{:clash_response => false, :smash_response => false}` by default.
+
+* Introduced `RC::Promise#future_response` to allow you customize the
+  behaviour of promises more easily.
+
+* Introduced `RC::Promise.claim` to allow you pre-fill a promise.
+
+* Introduced `RC::Promise#then` to allow you append a callback whenever
+  the response is ready. The type should be: `Env -> Response`
+
+* Now `RC::Promise#inspect` would show REQUEST_URI instead of REQUEST_PATH,
+  which should be easier to debug.
+
+* Introduced `RC::ThreadPool#size` which is a short hand for
+  `RC::ThreadPool#workers.size`.
+
+### Bugs fixed
+
+* Inheritance with `RC::Client` now works properly.
+* Now `RC::Cache` properly return cached headers.
+* Now `RC::Cache` would work more like `RC::Engine`, wrapping responses
+  inside futures.
+
 ## rest-core 3.1.1 -- 2014-05-13
 
 ### Enhancements

data/README.md

@@ -225,6 +225,26 @@ a large number of concurrent calls.
 Also, setting `pool_size` to `-1` would mean we want to make blocking
 requests, without spawning any threads. This might be useful for debugging.
 
+### Gracefully shutdown
+
+To shutdown gracefully, consider shutdown the thread pool (if we're using it),
+and wait for all requests for a given client. For example, suppose we're using
+`RC::Universal`, we'll do this when we're shutting down:
+
+``` ruby
+RC::Universal.shutdown
+```
+
+We could put them in `at_exit` callback like this:
+
+``` ruby
+at_exit do
+  RC::Universal.shutdown
+end
+```
+
+If you're using unicorn, you probably want to put that in the config.
+
 ### Persistent connections (keep-alive connections)
 
 Since we're using [httpclient][] by default now, we would reuse connections,
@@ -396,7 +416,10 @@ module RestCore
                                     # cache {} didn't support it
       use ErrorHandler, nil
       use ErrorDetectorHttp
-      use JsonResponse, false
+      use SmashResponse, false
+      use ClashResponse, false
+      use  JsonResponse, false
+      use QueryResponse, false
     end
   end
 end
@@ -493,6 +516,7 @@ simply ignore `:expires_in`.
 [RC::AuthBasic]: lib/rest-core/middleware/auth_basic.rb
 [RC::Bypass]: lib/rest-core/middleware/bypass.rb
 [RC::Cache]: lib/rest-core/middleware/cache.rb
+[RC::ClashResponse]: lib/rest-core/middleware/clash_response.rb
 [RC::CommonLogger]: lib/rest-core/middleware/common_logger.rb
 [RC::DefaultHeaders]: lib/rest-core/middleware/default_headers.rb
 [RC::DefaultPayload]: lib/rest-core/middleware/default_payload.rb
@@ -508,6 +532,7 @@ simply ignore `:expires_in`.
 [RC::Oauth1Header]: lib/rest-core/middleware/oauth1_header.rb
 [RC::Oauth2Header]: lib/rest-core/middleware/oauth2_header.rb
 [RC::Oauth2Query]: lib/rest-core/middleware/oauth2_query.rb
+[RC::SmashResponse]: lib/rest-core/middleware/smash_response.rb
 [RC::Timeout]: lib/rest-core/middleware/timeout.rb
 [moneta]: https://github.com/minad/moneta#expiration
 
@@ -663,6 +688,7 @@ and complete.
 
 ## rest-core users:
 
+* [rest-firebase](https://github.com/CodementorIO/rest-firebase)
 * [rest-more][]
 * [rest-more-yahoo_buy](https://github.com/GoodLife/rest-more-yahoo_buy)
 * [s2sync](https://github.com/brucehsu/s2sync)
@@ -671,6 +697,7 @@ and complete.
 
 ## Powered sites:
 
+* [Codementor](https://www.codementor.io/)
 * [PicCollage](http://pic-collage.com/)
 
 ## CHANGES:

data/TODO.md

@@ -3,10 +3,6 @@
 * middleware revisit (how to initialize?)
 * X-Method-Override
 
-# BUG
-
-* inheritance should work; assign builder?
-
 # FEATURE
 
 * middleware composer

data/lib/rest-core.rb

@@ -40,14 +40,19 @@ module RestCore
   # misc utilities
   autoload :Hmac          , 'rest-core/util/hmac'
   autoload :Json          , 'rest-core/util/json'
+  autoload :ParseLink     , 'rest-core/util/parse_link'
   autoload :ParseQuery    , 'rest-core/util/parse_query'
   autoload :Payload       , 'rest-core/util/payload'
   autoload :Config        , 'rest-core/util/config'
+  autoload :Clash         , 'rest-core/util/clash'
+  autoload :Smash         , 'rest-core/util/smash'
 
   # middlewares
   autoload :AuthBasic     , 'rest-core/middleware/auth_basic'
   autoload :Bypass        , 'rest-core/middleware/bypass'
   autoload :Cache         , 'rest-core/middleware/cache'
+  autoload :ClashResponse , 'rest-core/middleware/clash_response'
+  autoload :SmashResponse , 'rest-core/middleware/smash_response'
   autoload :CommonLogger  , 'rest-core/middleware/common_logger'
   autoload :DefaultHeaders, 'rest-core/middleware/default_headers'
   autoload :DefaultQuery  , 'rest-core/middleware/default_query'

data/lib/rest-core/builder.rb

@@ -45,6 +45,16 @@ class RestCore::Builder
     Module.new do
       attr_accessor :builder, :pool_size, :pool_idle_time,
                     :event_source_class, :promises, :mutex
+
+      def inherited sub
+        sub.builder            = builder
+        sub.pool_size          = pool_size
+        sub.pool_idle_time     = pool_idle_time
+        sub.event_source_class = event_source_class
+        sub.promises           = []
+        sub.mutex              = Mutex.new
+      end
+
       def thread_pool; RestCore::ThreadPool[self]; end
 
       def give_promise weak_promise, ps=promises, m=mutex
@@ -54,6 +64,13 @@ class RestCore::Builder
         end
       end
 
+      # Shutdown the thread pool for this client and wait for all requests
+      def shutdown
+        thread_pool.shutdown
+        wait
+      end
+
+      # Wait for all the requests to be done for this client
       def wait ps=promises, m=mutex
         return self if ps.empty?
         current_promises = nil

data/lib/rest-core/client/universal.rb

@@ -12,10 +12,13 @@ module RestCore
 
     use FollowRedirect, 10
     use CommonLogger  , method(:puts)
-    use Cache         ,  {}, 600 do
+    use Cache         ,  {}, 600 do # default :expires_in 600 but the default
+                                    # cache {} didn't support it
       use ErrorHandler, nil
       use ErrorDetectorHttp
-      use JsonResponse, false
+      use SmashResponse, false
+      use ClashResponse, false
+      use  JsonResponse, false
       use QueryResponse, false
     end
   end

data/lib/rest-core/engine.rb

@@ -9,12 +9,7 @@ class RestCore::Engine
     req     = env.merge(REQUEST_URI => request_uri(env))
     promise = Promise.new(req, k, req[ASYNC])
     promise.defer{ request(promise, req) }
-    req.merge(RESPONSE_BODY    => promise.future_body,
-              RESPONSE_STATUS  => promise.future_status,
-              RESPONSE_HEADERS => promise.future_headers,
-              RESPONSE_SOCKET  => promise.future_socket,
-              FAIL             => promise.future_failures,
-              PROMISE          => promise)
+    promise.future_response
   end
 
   private

data/lib/rest-core/middleware/cache.rb

@@ -22,18 +22,16 @@ class RestCore::Cache
           env
         end
 
-    if cached = cache_get(e)
+    cache_get(e){ |cached|
       e[TIMER].cancel if e[TIMER]
       wrapped.call(cached, &k)
-    else
-      app.call(e){ |res|
-        wrapped.call(res){ |res_wrapped|
-          k.call(if (res_wrapped[FAIL] || []).empty?
-                   cache_for(res).merge(res_wrapped)
-                 else
-                   res_wrapped
-                 end)}}
-    end
+    } || app.call(e){ |res|
+           wrapped.call(res){ |res_wrapped|
+             k.call(if (res_wrapped[FAIL] || []).empty?
+                      cache_for(res).merge(res_wrapped)
+                    else
+                      res_wrapped
+                    end)}}
   end
 
   def cache_key env
@@ -41,14 +39,15 @@ class RestCore::Cache
                                              cache_key_raw(env))}"
   end
 
-  def cache_get env
+  def cache_get env, &k
     return unless cache(env)
     return unless cache_for?(env)
 
+    uri = request_uri(env)
     start_time = Time.now
     return unless data = cache(env)[cache_key(env)]
-    log(env, Event::CacheHit.new(Time.now - start_time, request_uri(env))).
-      merge(data_extract(data))
+    log(env, Event::CacheHit.new(Time.now - start_time, uri)).
+      merge(data_extract(data, env.merge(REQUEST_URI => uri), k))
   end
 
   private
@@ -96,13 +95,12 @@ class RestCore::Cache
     "#{ res[RESPONSE_BODY]}"
   end
 
-  def data_extract data
+  def data_extract data, env, k
     _, status, headers, body =
-      data.match(/\A(\d+)\n([^\n]+\n)*\n\n(.*)\Z/m).to_a
+      data.match(/\A(\d+)\n((?:[^\n]+\n)*)\n\n(.*)\Z/m).to_a
 
-    {RESPONSE_BODY    => body,
-     RESPONSE_HEADERS => Hash[(headers||'').scan(/([^:]+): ([^\n]+)\n/)],
-     RESPONSE_STATUS  => status.to_i}
+    Promise.claim(env, k, body,status.to_i,
+      Hash[(headers||'').scan(/([^:]+): ([^\n]+)\n/)]).future_response
   end
 
   def cache_for? env

data/lib/rest-core/middleware/clash_response.rb

@@ -0,0 +1,21 @@
+
+require 'rest-core/middleware'
+require 'rest-core/util/clash'
+
+class RestCore::ClashResponse
+  def self.members; [:clash_response]; end
+  include RestCore::Middleware
+
+  def call env, &k
+    return app.call(env, &k) if env[DRY]
+    return app.call(env, &k) unless clash_response(env)
+
+    app.call(env){ |res|
+      if res[RESPONSE_BODY].kind_of?(Hash)
+        yield(res.merge(RESPONSE_BODY => Clash.new(res[RESPONSE_BODY])))
+      else
+        yield(res)
+      end
+    }
+  end
+end

data/lib/rest-core/middleware/json_response.rb

@@ -6,6 +6,14 @@ class RestCore::JsonResponse
   def self.members; [:json_response]; end
   include RestCore::Middleware
 
+  class ParseError < Json.const_get(:ParseError)
+    attr_reader :cause, :body
+    def initialize cause, body
+      super("#{cause.message}\nOriginal text: #{body}")
+      @cause, @body = cause, body
+    end
+  end
+
   JSON_RESPONSE_HEADER = {'Accept' => 'application/json'}.freeze
 
   def call env, &k
@@ -19,10 +27,10 @@ class RestCore::JsonResponse
   end
 
   def process response
-    response.merge(RESPONSE_BODY =>
-      Json.decode("[#{response[RESPONSE_BODY]}]").first)
-      # [this].first is not needed for yajl-ruby
+    body = response[RESPONSE_BODY]
+    response.merge(RESPONSE_BODY => Json.decode("[#{body}]").first)
+    # [this].first is not needed for yajl-ruby
   rescue Json.const_get(:ParseError) => error
-    fail(response, error)
+    fail(response, ParseError.new(error, body))
   end
 end

data/lib/rest-core/middleware/smash_response.rb

@@ -0,0 +1,21 @@
+
+require 'rest-core/middleware'
+require 'rest-core/util/smash'
+
+class RestCore::SmashResponse
+  def self.members; [:smash_response]; end
+  include RestCore::Middleware
+
+  def call env, &k
+    return app.call(env, &k) if env[DRY]
+    return app.call(env, &k) unless smash_response(env)
+
+    app.call(env){ |res|
+      if res[RESPONSE_BODY].kind_of?(Hash)
+        yield(res.merge(RESPONSE_BODY => Smash.new(res[RESPONSE_BODY])))
+      else
+        yield(res)
+      end
+    }
+  end
+end

data/lib/rest-core/promise.rb

@@ -15,9 +15,15 @@ class RestCore::Promise
     end
   end
 
+  def self.claim env, k=RC.id, body, status, headers
+    promise = new(env, k)
+    promise.fulfill(body, status, headers)
+    promise
+  end
+
   def initialize env, k=RC.id, immediate=false, &job
     self.env       = env
-    self.k         = k
+    self.k         = [k]
     self.immediate = immediate
 
     self.body, self.status, self.headers,
@@ -30,7 +36,7 @@ class RestCore::Promise
   end
 
   def inspect
-    "<#{self.class.name} for #{env[REQUEST_PATH]}>"
+    "<#{self.class.name} for #{env[REQUEST_URI]}>"
   end
 
   def future_body    ; Future.new(self, RESPONSE_BODY   ); end
@@ -38,6 +44,14 @@ class RestCore::Promise
   def future_headers ; Future.new(self, RESPONSE_HEADERS); end
   def future_socket  ; Future.new(self, RESPONSE_SOCKET ); end
   def future_failures; Future.new(self, FAIL)            ; end
+  def future_response
+    env.merge(RESPONSE_BODY    => future_body,
+              RESPONSE_STATUS  => future_status,
+              RESPONSE_HEADERS => future_headers,
+              RESPONSE_SOCKET  => future_socket,
+              FAIL             => future_failures,
+              PROMISE          => self)
+  end
 
   # called in client thread
   def defer &job
@@ -87,6 +101,12 @@ class RestCore::Promise
     fulfill('', 0, {})
   end
 
+  # append your actions, which would be called when we're calling back
+  def then &action
+    k << action
+    self
+  end
+
   protected
   attr_accessor :env, :k, :immediate,
                 :response, :body, :status, :headers, :socket, :error,
@@ -107,13 +127,13 @@ class RestCore::Promise
 
   # called in client thread, when yield is called
   def callback
-    self.response ||= k.call(
+    self.response ||= k.inject(
       env.merge(RESPONSE_BODY    => body  ,
                 RESPONSE_STATUS  => status,
                 RESPONSE_HEADERS => headers,
                 RESPONSE_SOCKET  => socket,
                 FAIL             => ((env[FAIL]||[]) + [error]).compact,
-                LOG              =>   env[LOG] ||[]))
+                LOG              =>   env[LOG] ||[])){ |r, i| i.call(r) }
   end
 
   # called in requesting thread, whenever the request is done

data/lib/rest-core/thread_pool.rb

@@ -74,6 +74,10 @@ class RestCore::ThreadPool
     "#<#{self.class.name} client_class=#{client_class}>"
   end
 
+  def size
+    workers.size
+  end
+
   def max_size
     client_class.pool_size
   end

data/lib/rest-core/timer.rb

@@ -2,7 +2,11 @@
 require 'timers'
 
 class RestCore::Timer
-  TimerGen = Timers.new
+  TimerGen = if Timers.respond_to?(:new)
+               Timers.new
+             else
+               Timers::Group.new
+             end
 
   attr_accessor :timeout, :error
   def initialize timeout, error, &block

data/lib/rest-core/util/clash.rb

@@ -0,0 +1,50 @@
+
+module RestCore; end
+class RestCore::Clash
+  Empty = Hash.new(&(l = lambda{|_,_|Hash.new(&l).freeze})).freeze
+
+  attr_accessor :data
+  def initialize data
+    self.data = data
+  end
+
+  def [] k
+    if data.key?(k)
+      convert(data[k])
+    else
+      Empty
+    end
+  end
+
+  def == rhs
+    if rhs.kind_of?(RestCore::Clash)
+      data == rhs.data
+    else
+      data == rhs
+    end
+  end
+
+  private
+  def convert value
+    case value
+    when Hash
+      RestCore::Clash.new(value)
+    when Array
+      value.map{ |ele| convert(ele) }
+    else
+      value
+    end
+  end
+
+  def respond_to_missing? msg, include_private=false
+    data.respond_to?(msg, include_private)
+  end
+
+  def method_missing msg, *args, &block
+    if data.respond_to?(msg)
+      data.public_send(msg, *args, &block)
+    else
+      super
+    end
+  end
+end

data/lib/rest-core/util/parse_link.rb

@@ -0,0 +1,16 @@
+
+module RestCore; end
+module RestCore::ParseLink
+  module_function
+  # http://tools.ietf.org/html/rfc5988
+  parname = '"?([^"]+)"?'
+  LINKPARAM = /#{parname}=#{parname}/
+  def parse_link link
+    link.split(',').inject({}) do |r, value|
+      uri, *pairs = value.split(';')
+      params = Hash[pairs.map{ |p| p.strip.match(LINKPARAM)[1..2] }]
+      r[params['rel']] = params.merge('uri' => uri[/<([^>]+)>/, 1])
+      r
+    end
+  end
+end

data/lib/rest-core/util/smash.rb

@@ -0,0 +1,41 @@
+
+module RestCore; end
+class RestCore::Smash
+  attr_accessor :data
+  def initialize data
+    self.data = data
+  end
+
+  def [] *keys
+    keys.inject(data) do |r, k|
+      if r.respond_to?(:key) && r.key?(k)
+        r[k]
+      elsif r.respond_to?(:[])
+        r[k]
+      else
+        return nil # stop here
+      end
+    end
+  end
+
+  def == rhs
+    if rhs.kind_of?(RestCore::Smash)
+      data == rhs.data
+    else
+      data == rhs
+    end
+  end
+
+  private
+  def respond_to_missing? msg, include_private=false
+    data.respond_to?(msg, include_private)
+  end
+
+  def method_missing msg, *args, &block
+    if data.respond_to?(msg)
+      data.public_send(msg, *args, &block)
+    else
+      super
+    end
+  end
+end

data/lib/rest-core/version.rb

@@ -1,4 +1,4 @@
 
 module RestCore
-  VERSION = '3.1.1'
+  VERSION = '3.2.0'
 end

data/rest-core.gemspec

@@ -1,14 +1,14 @@
 # -*- encoding: utf-8 -*-
-# stub: rest-core 3.1.1 ruby lib
+# stub: rest-core 3.2.0 ruby lib
 
 Gem::Specification.new do |s|
   s.name = "rest-core"
-  s.version = "3.1.1"
+  s.version = "3.2.0"
 
   s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
   s.require_paths = ["lib"]
   s.authors = ["Lin Jen-Shin (godfat)"]
-  s.date = "2014-05-13"
+  s.date = "2014-06-27"
   s.description = "Modular Ruby clients interface for REST APIs.\n\nThere has been an explosion in the number of REST APIs available today.\nTo address the need for a way to access these APIs easily and elegantly,\nwe have developed rest-core, which consists of composable middleware\nthat allows you to build a REST client for any REST API. Or in the case of\ncommon APIs such as Facebook, Github, and Twitter, you can simply use the\ndedicated clients provided by [rest-more][].\n\n[rest-more]: https://github.com/godfat/rest-more"
   s.email = ["godfat (XD) godfat.org"]
   s.files = [
@@ -42,6 +42,7 @@ Gem::Specification.new do |s|
   "lib/rest-core/middleware/auth_basic.rb",
   "lib/rest-core/middleware/bypass.rb",
   "lib/rest-core/middleware/cache.rb",
+  "lib/rest-core/middleware/clash_response.rb",
   "lib/rest-core/middleware/common_logger.rb",
   "lib/rest-core/middleware/default_headers.rb",
   "lib/rest-core/middleware/default_payload.rb",
@@ -58,6 +59,7 @@ Gem::Specification.new do |s|
   "lib/rest-core/middleware/oauth2_header.rb",
   "lib/rest-core/middleware/oauth2_query.rb",
   "lib/rest-core/middleware/query_response.rb",
+  "lib/rest-core/middleware/smash_response.rb",
   "lib/rest-core/middleware/timeout.rb",
   "lib/rest-core/patch/multi_json.rb",
   "lib/rest-core/patch/rest-client.rb",
@@ -65,11 +67,14 @@ Gem::Specification.new do |s|
   "lib/rest-core/test.rb",
   "lib/rest-core/thread_pool.rb",
   "lib/rest-core/timer.rb",
+  "lib/rest-core/util/clash.rb",
   "lib/rest-core/util/config.rb",
   "lib/rest-core/util/hmac.rb",
   "lib/rest-core/util/json.rb",
+  "lib/rest-core/util/parse_link.rb",
   "lib/rest-core/util/parse_query.rb",
   "lib/rest-core/util/payload.rb",
+  "lib/rest-core/util/smash.rb",
   "lib/rest-core/version.rb",
   "lib/rest-core/wrapper.rb",
   "rest-core.gemspec",
@@ -79,6 +84,8 @@ Gem::Specification.new do |s|
   "test/test_auth_basic.rb",
   "test/test_builder.rb",
   "test/test_cache.rb",
+  "test/test_clash.rb",
+  "test/test_clash_response.rb",
   "test/test_client.rb",
   "test/test_client_oauth1.rb",
   "test/test_config.rb",
@@ -94,11 +101,14 @@ Gem::Specification.new do |s|
   "test/test_json_response.rb",
   "test/test_oauth1_header.rb",
   "test/test_oauth2_header.rb",
+  "test/test_parse_link.rb",
   "test/test_payload.rb",
   "test/test_promise.rb",
   "test/test_query_response.rb",
   "test/test_rest-client.rb",
   "test/test_simple.rb",
+  "test/test_smash.rb",
+  "test/test_smash_response.rb",
   "test/test_thread_pool.rb",
   "test/test_timeout.rb",
   "test/test_universal.rb",
@@ -111,6 +121,8 @@ Gem::Specification.new do |s|
   "test/test_auth_basic.rb",
   "test/test_builder.rb",
   "test/test_cache.rb",
+  "test/test_clash.rb",
+  "test/test_clash_response.rb",
   "test/test_client.rb",
   "test/test_client_oauth1.rb",
   "test/test_config.rb",
@@ -126,11 +138,14 @@ Gem::Specification.new do |s|
   "test/test_json_response.rb",
   "test/test_oauth1_header.rb",
   "test/test_oauth2_header.rb",
+  "test/test_parse_link.rb",
   "test/test_payload.rb",
   "test/test_promise.rb",
   "test/test_query_response.rb",
   "test/test_rest-client.rb",
   "test/test_simple.rb",
+  "test/test_smash.rb",
+  "test/test_smash_response.rb",
   "test/test_thread_pool.rb",
   "test/test_timeout.rb",
   "test/test_universal.rb",

data/test/test_cache.rb

@@ -7,6 +7,18 @@ describe RC::Cache do
     Muack.verify
   end
 
+  def simple_client
+    RC::Builder.client{ use RC::Cache, {}, nil }.new
+  end
+
+  def json_client
+    RC::Builder.client do
+      use RC::Cache, {}, 3600 do
+        use RC::JsonResponse, true
+      end
+    end.new
+  end
+
   should 'basic 0' do
     c = RC::Builder.client do
       use RC::Cache, {}, 3600
@@ -60,17 +72,13 @@ describe RC::Cache do
   should 'not raise error if headers is nil' do
     path = 'http://a'
     stub_request(:get , path).to_return(:body => 'OK', :headers => nil)
-    c = RC::Builder.client do
-      use RC::Cache, {}, nil
-    end.new
+    c = simple_client
     c.get(path).should.eq 'OK'
     c.get(path).should.eq 'OK'
   end
 
   should 'head then get' do
-    c = RC::Builder.client do
-      use RC::Cache, {}, nil
-    end.new
+    c = simple_client
     path = 'http://example.com'
     stub_request(:head, path).to_return(:headers => {'A' => 'B'})
     c.head(path).should.eq('A' => 'B')
@@ -98,20 +106,40 @@ describe RC::Cache do
   end
 
   should 'cache the original response' do
-    c = RC::Builder.client do
-      use RC::Cache, {}, 3600 do
-        use RC::JsonResponse, true
-      end
-    end.new
+    c = json_client
     stub_request(:get, 'http://me').to_return(:body => body = '{"a":"b"}')
     c.get('http://me').should.eq 'a' => 'b'
     c.cache.values.first.should.eq "200\n\n\n#{body}"
   end
 
+  should 'cache multiple headers' do
+    c = simple_client
+    stub_request(:get, 'http://me').to_return(:headers =>
+      {'Apple' => 'Orange', 'Orange' => 'Apple'})
+    expected = {'APPLE' => 'Orange', 'ORANGE' => 'Apple'}
+    args = ['http://me', {}, {RC::RESPONSE_KEY => RC::RESPONSE_HEADERS}]
+    2.times{ c.get(*args).should.eq expected }
+  end
+
+  should 'preserve promise and REQUEST_URI' do
+    c = simple_client
+    uri = 'http://me?a=b'
+    stub_request(:get, uri)
+    args = ['http://me', {:a => 'b'}, {RC::RESPONSE_KEY => RC::PROMISE}]
+    2.times{ c.get(*args).yield[RC::REQUEST_URI].should.eq uri }
+  end
+
+  should 'preserve promise and preserve wrapped call' do
+    c = json_client
+    stub_request(:get, 'http://me').to_return(:body => body = '{"a":"b"}')
+    args = ['http://me', {}, {RC::RESPONSE_KEY => RC::PROMISE}]
+    2.times do
+      c.get(*args).then{ |r| r[RC::RESPONSE_BODY].should.eq 'a' => 'b' }.yield
+    end
+  end
+
   should 'multiline response' do
-    c = RC::Builder.client do
-      use RC::Cache, {}, 3600
-    end.new
+    c = simple_client
     stub_request(:get, 'http://html').to_return(:body => body = "a\n\nb")
     c.get('http://html').should.eq body
     c.cache.values.first.should.eq "200\n\n\n#{body}"
@@ -151,14 +179,14 @@ describe RC::Cache do
   end
 
   should 'not cache dry run' do
-    c = RC::Builder.client{use RC::Cache, {}, nil}.new
+    c = simple_client
     c.url('test')
     c.cache.should.eq({})
   end
 
   should 'not cache hijacking' do
     stub_request(:get, 'http://a').to_return(:body => 'ok')
-    c = RC::Builder.client{use RC::Cache, {}, nil}.new
+    c = simple_client
     2.times do
       c.get('http://a', {}, RC::HIJACK => true,
                             RC::RESPONSE_KEY => RC::RESPONSE_SOCKET).
@@ -170,7 +198,7 @@ describe RC::Cache do
   should 'update cache if there is cache option set to false' do
     url, body = "https://cache", 'ok'
     stub_request(:get, url).to_return(:body => body)
-    c = RC::Builder.client{use RC::Cache, {}, nil}.new
+    c = simple_client
 
     c.get(url)                            .should.eq body
     stub_request(:get, url).to_return(:body => body.reverse).times(2)

data/test/test_clash.rb

@@ -0,0 +1,24 @@
+
+require 'rest-core/test'
+
+describe RC::Clash do
+  should 'never give nil for non-existing values' do
+    h = {0 => 1, 2 => {3 => 4, 5 => [6, {7 => 8}]}, 9 => false, 10 => nil}
+    c = RC::Clash.new(h)
+    c[0]            .should.eq(1)
+    c[1]            .should.eq({})
+    c[1][2]         .should.eq({})
+    c[1][2][3]      .should.eq({})
+    c[2]            .should.eq(3 => 4, 5 => [6, {7 => 8}])
+    c[2][3]         .should.eq(4)
+    c[2][4]         .should.eq({})
+    c[2][4][5]      .should.eq({})
+    c[2][5]         .should.eq([6, {7 => 8}])
+    c[2][5][1]      .should.eq(7 => 8)
+    c[2][5][1][7]   .should.eq(8)
+    c[2][5][1][8]   .should.eq({})
+    c[2][5][1][8][9].should.eq({})
+    c[9]            .should.eq(false)
+    c[10]           .should.eq(nil)
+  end
+end

data/test/test_clash_response.rb

@@ -0,0 +1,56 @@
+
+require 'rest-core/test'
+
+describe RC::ClashResponse do
+  describe 'app' do
+    app = RC::ClashResponse.new(RC::Dry.new, true)
+
+    should 'do nothing' do
+      env = {RC::RESPONSE_BODY => []}
+      app.call(env) do |res|
+        res.should.eq(env)
+        res[RC::RESPONSE_BODY].should.kind_of(Array)
+      end
+    end
+
+    should 'clash' do
+      app.call(RC::RESPONSE_BODY => {}) do |res|
+        body = res[RC::RESPONSE_BODY]
+        body.should.kind_of(RC::Clash)
+        body.should.empty
+        body[0].should.eq({})
+        body[0][0].should.eq({})
+      end
+    end
+
+    describe 'client' do
+      body = {0 => {1 => 2}}
+      client = RC::Builder.client do
+        use RC::ClashResponse, true
+        run Class.new{
+          define_method(:call) do |env, &block|
+            block.call(env.merge(RC::RESPONSE_BODY => body))
+          end
+        }
+      end
+
+      should 'do nothing' do
+        b = client.new(:clash_response => false).get(''){ |res|
+          res.should.eq(body)
+          res.should.kind_of(Hash)
+        }.get('')
+        b.should.eq(body)
+        b.should.kind_of(Hash)
+      end
+
+      should 'clash' do
+        b = client.new.get(''){ |res|
+          res.should.eq(body)
+          res.should.kind_of(RC::Clash)
+        }.get('')
+        b.should.eq(body)
+        b.should.kind_of(RC::Clash)
+      end
+    end
+  end
+end

data/test/test_client.rb

@@ -67,13 +67,13 @@ describe RC::Simple do
     stub(any_instance_of(WeakRef)).weakref_alive?{false}
     client.new.get(url)
     client.promises.size.should < 6
-    client.wait
+    client.shutdown
     client.promises.should.empty
   end
 
   should 'have correct to_i' do
-    stub_request(:get, 'http://localhost/').to_return(:body => '123')
-    RC::Simple.new.get('http://localhost/').to_i.should.eq 123
+    stub_request(:get, url).to_return(:body => '123')
+    RC::Simple.new.get(url).to_i.should.eq 123
   end
 
   should 'use defaults' do
@@ -106,4 +106,9 @@ describe RC::Simple do
     c.timeout = false
     c.timeout.should.eq false # false would disable default
   end
+
+  should 'work for inheritance' do
+    stub_request(:get, url).to_return(:body => '123')
+    Class.new(RC::Simple).new.get(url).should.eq '123'
+  end
 end

data/test/test_json_response.rb

@@ -4,6 +4,7 @@ require 'rest-core/test'
 describe RC::JsonResponse do
   describe 'app' do
     app = RC::JsonResponse.new(RC::Dry.new, true)
+    bad = 'bad json'
 
     should 'do nothing' do
       expected = {RC::RESPONSE_BODY => nil,
@@ -14,9 +15,26 @@ describe RC::JsonResponse do
     should 'decode' do
       expected = {RC::RESPONSE_BODY => {},
                   RC::REQUEST_HEADERS => {'Accept' => 'application/json'}}
-      app.call(RC::RESPONSE_BODY => '{}'){ |response|
+      app.call(RC::RESPONSE_BODY => '{}') do |response|
         response.should.eq(expected)
-      }
+      end
+    end
+
+    should 'give proper parse error' do
+      app.call(RC::RESPONSE_BODY => bad) do |response|
+        err = response[RC::FAIL].first
+        err.should.kind_of(RC::Json.const_get(:ParseError))
+        err.should.kind_of(RC::JsonResponse::ParseError)
+      end
+    end
+
+    should 'give me original text' do
+      app.call(RC::RESPONSE_BODY => bad) do |response|
+        err = response[RC::FAIL].first
+        err.message    .should.include(bad)
+        err.body       .should.eq(bad)
+        err.cause.class.should.eq(RC::Json.const_get(:ParseError))
+      end
     end
   end
 

data/test/test_parse_link.rb

@@ -0,0 +1,43 @@
+
+require 'rest-core/test'
+
+describe RC::ParseLink do
+  describe 'http://tools.ietf.org/html/rfc5988' do
+    should '5.5 a' do
+      link = '<http://example.com/TheBook/chapter2>; rel="previous"; title="previous chapter"'
+      RC::ParseLink.parse_link(link).should.eq(
+        'previous' => {'uri' => 'http://example.com/TheBook/chapter2',
+                       'rel' => 'previous',
+                       'title' => 'previous chapter'})
+    end
+
+    should '5.5 b' do
+      link = '</>; rel="http://example.net/foo"'
+      RC::ParseLink.parse_link(link).should.eq(
+        'http://example.net/foo' => {'uri' => '/',
+                                     'rel' => 'http://example.net/foo'})
+    end
+
+    should '5.5 c (we did not implement * and unescape for now)' do
+      link = <<-LINK
+</TheBook/chapter2>; rel="previous"; title*=UTF-8'de'letztes%20Kapitel, </TheBook/chapter4>; rel="next"; title*=UTF-8'de'n%c3%a4chstes%20Kapitel
+      LINK
+      RC::ParseLink.parse_link(link).should.eq(
+        'previous' => {'uri' => '/TheBook/chapter2',
+                       'rel' => 'previous',
+                       'title*' => "UTF-8'de'letztes%20Kapitel"},
+        'next'     => {'uri' => '/TheBook/chapter4',
+                       'rel' => 'next',
+                       'title*' => "UTF-8'de'n%c3%a4chstes%20Kapitel"})
+    end
+
+    should '5.5 d' do
+      link = '<http://example.org/>; rel="start http://example.net/relation/other"'
+
+      RC::ParseLink.parse_link(link).should.eq(
+        'start http://example.net/relation/other' =>
+          {'uri' => 'http://example.org/',
+           'rel' => 'start http://example.net/relation/other'})
+    end
+  end
+end

data/test/test_promise.rb

@@ -12,7 +12,8 @@ describe RC::Promise do
   end
 
   after do
-    @client.thread_pool.shutdown
+    @client.shutdown
+    @client.thread_pool.size.should.eq 0
     Muack.verify
   end
 
@@ -44,6 +45,15 @@ describe RC::Promise do
     @promise.send(:headers).should.eq('K' => 'V')
   end
 
+  should 'then then then' do
+    plusone = lambda do |r|
+      r.merge(RC::RESPONSE_BODY => r[RC::RESPONSE_BODY] + 1)
+    end
+    2.times{ @promise.then(&plusone).then(&plusone).then(&plusone) }
+    @promise.fulfill(0, 200, {})
+    @promise.future_body.should.eq 6
+  end
+
   should 'call inline if pool_size < 0' do
     @client.pool_size = -1
     current_thread = Thread.current

data/test/test_smash.rb

@@ -0,0 +1,24 @@
+
+require 'rest-core/test'
+
+describe RC::Smash do
+  should 'deep access' do
+    h = {0 => 1, 2 => {3 => 4, 5 => [6, {7 => 8}]}, 9 => false, 10 => nil}
+    c = RC::Smash.new(h)
+    c[0]            .should.eq(1)
+    c[1]            .should.eq(nil)
+    c[1, 2]         .should.eq(nil)
+    c[1, 2, 3]      .should.eq(nil)
+    c[2]            .should.eq(3 => 4, 5 => [6, {7 => 8}])
+    c[2, 3]         .should.eq(4)
+    c[2, 4]         .should.eq(nil)
+    c[2, 4, 5]      .should.eq(nil)
+    c[2, 5]         .should.eq([6, {7 => 8}])
+    c[2, 5, 1]      .should.eq(7 => 8)
+    c[2, 5, 1, 7]   .should.eq(8)
+    c[2, 5, 1, 8]   .should.eq(nil)
+    c[2, 5, 1, 8, 9].should.eq(nil)
+    c[9]            .should.eq(false)
+    c[10]           .should.eq(nil)
+  end
+end

data/test/test_smash_response.rb

@@ -0,0 +1,56 @@
+
+require 'rest-core/test'
+
+describe RC::SmashResponse do
+  describe 'app' do
+    app = RC::SmashResponse.new(RC::Dry.new, true)
+
+    should 'do nothing' do
+      env = {RC::RESPONSE_BODY => []}
+      app.call(env) do |res|
+        res.should.eq(env)
+        res[RC::RESPONSE_BODY].should.kind_of(Array)
+      end
+    end
+
+    should 'smash' do
+      app.call(RC::RESPONSE_BODY => {}) do |res|
+        body = res[RC::RESPONSE_BODY]
+        body.should.kind_of(RC::Smash)
+        body.should.empty
+        body[0].should.eq(nil)
+        body[0, 0].should.eq(nil)
+      end
+    end
+
+    describe 'client' do
+      body = {0 => {1 => 2}}
+      client = RC::Builder.client do
+        use RC::SmashResponse, true
+        run Class.new{
+          define_method(:call) do |env, &block|
+            block.call(env.merge(RC::RESPONSE_BODY => body))
+          end
+        }
+      end
+
+      should 'do nothing' do
+        b = client.new(:smash_response => false).get(''){ |res|
+          res.should.eq(body)
+          res.should.kind_of(Hash)
+        }.get('')
+        b.should.eq(body)
+        b.should.kind_of(Hash)
+      end
+
+      should 'clash' do
+        b = client.new.get(''){ |res|
+          res.should.eq(body)
+          res.should.kind_of(RC::Smash)
+        }.get('')
+        b.should.eq(body)
+        b.should.kind_of(RC::Smash)
+      end
+    end
+  end
+end

data/test/test_universal.rb

@@ -16,4 +16,13 @@ describe RC::Universal do
     u.request_full(env, u.dry)[RC::REQUEST_HEADERS].should.eq(
       {'Authorization' => 'Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ=='}.merge(acc))
   end
+
+  should 'clash' do
+    url = 'http://localhost/'
+    stub_request(:get, url).to_return(:body => '{"a":{"b":"c"}}')
+    res = RC::Universal.new(:json_response => true,
+                            :clash_response => true,
+                            :log_method => false).get(url)
+    res['a']['d'].should.eq({})
+  end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: rest-core
 version: !ruby/object:Gem::Version
-  version: 3.1.1
+  version: 3.2.0
 platform: ruby
 authors:
 - Lin Jen-Shin (godfat)
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2014-05-13 00:00:00.000000000 Z
+date: 2014-06-27 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: httpclient
@@ -99,6 +99,7 @@ files:
 - lib/rest-core/middleware/auth_basic.rb
 - lib/rest-core/middleware/bypass.rb
 - lib/rest-core/middleware/cache.rb
+- lib/rest-core/middleware/clash_response.rb
 - lib/rest-core/middleware/common_logger.rb
 - lib/rest-core/middleware/default_headers.rb
 - lib/rest-core/middleware/default_payload.rb
@@ -115,6 +116,7 @@ files:
 - lib/rest-core/middleware/oauth2_header.rb
 - lib/rest-core/middleware/oauth2_query.rb
 - lib/rest-core/middleware/query_response.rb
+- lib/rest-core/middleware/smash_response.rb
 - lib/rest-core/middleware/timeout.rb
 - lib/rest-core/patch/multi_json.rb
 - lib/rest-core/patch/rest-client.rb
@@ -122,11 +124,14 @@ files:
 - lib/rest-core/test.rb
 - lib/rest-core/thread_pool.rb
 - lib/rest-core/timer.rb
+- lib/rest-core/util/clash.rb
 - lib/rest-core/util/config.rb
 - lib/rest-core/util/hmac.rb
 - lib/rest-core/util/json.rb
+- lib/rest-core/util/parse_link.rb
 - lib/rest-core/util/parse_query.rb
 - lib/rest-core/util/payload.rb
+- lib/rest-core/util/smash.rb
 - lib/rest-core/version.rb
 - lib/rest-core/wrapper.rb
 - rest-core.gemspec
@@ -136,6 +141,8 @@ files:
 - test/test_auth_basic.rb
 - test/test_builder.rb
 - test/test_cache.rb
+- test/test_clash.rb
+- test/test_clash_response.rb
 - test/test_client.rb
 - test/test_client_oauth1.rb
 - test/test_config.rb
@@ -151,11 +158,14 @@ files:
 - test/test_json_response.rb
 - test/test_oauth1_header.rb
 - test/test_oauth2_header.rb
+- test/test_parse_link.rb
 - test/test_payload.rb
 - test/test_promise.rb
 - test/test_query_response.rb
 - test/test_rest-client.rb
 - test/test_simple.rb
+- test/test_smash.rb
+- test/test_smash_response.rb
 - test/test_thread_pool.rb
 - test/test_timeout.rb
 - test/test_universal.rb
@@ -188,6 +198,8 @@ test_files:
 - test/test_auth_basic.rb
 - test/test_builder.rb
 - test/test_cache.rb
+- test/test_clash.rb
+- test/test_clash_response.rb
 - test/test_client.rb
 - test/test_client_oauth1.rb
 - test/test_config.rb
@@ -203,11 +215,14 @@ test_files:
 - test/test_json_response.rb
 - test/test_oauth1_header.rb
 - test/test_oauth2_header.rb
+- test/test_parse_link.rb
 - test/test_payload.rb
 - test/test_promise.rb
 - test/test_query_response.rb
 - test/test_rest-client.rb
 - test/test_simple.rb
+- test/test_smash.rb
+- test/test_smash_response.rb
 - test/test_thread_pool.rb
 - test/test_timeout.rb
 - test/test_universal.rb