rest-core 0.8.2 → 1.0.0

data/.travis.yml

@@ -1,4 +1,5 @@
-script: 'git submodule update --init; bundle exec rake test'
+before_install: 'git submodule update --init'
+script: 'bundle exec rake test'
 
 rvm:
   - 1.8.7
@@ -6,4 +7,5 @@ rvm:
   - 1.9.3
   - rbx-18mode
   - rbx-19mode
-  - jruby
+  - jruby-18mode
+  - jruby-19mode

data/CHANGES.md

@@ -1,33 +1,143 @@
 # CHANGES
 
+## rest-core 1.0.0 -- 2012-03-17
+
+This is a very significant release. The most important change is now we
+support asynchronous requests, by either passing a callback block or using
+fibers in Ruby 1.9 to make the whole program still look synchronous.
+
+Please read [README.md](https://github.com/cardinalblue/rest-core/blob/master/README.md)
+or [example](https://github.com/cardinalblue/rest-core/blob/master/example)
+for more detail.
+
+* [`Client`] Client#inspect is fixed for clients which do not have any
+  attributes.
+
+* [`Client`] HEAD, OPTIONS, and PATCH requests are added. For example:
+
+  ``` ruby
+      client = Client.new
+      client.head('path')
+      client.options('path')
+      client.patch('path')
+  ```
+
+* [`Client`] Now if you passed a block to either `get` or `post` or other
+  requests, the response would be returned to the block instead the caller.
+  In this case, the return value of `get` or `post` would be the client
+  itself. For example:
+
+  ``` ruby
+      client = Client.new
+      client.get('path'){ |response| puts response.insepct }.
+             get('math'){ |response| puts response.insepct }
+  ```
+
+* [`RestClient`] Now all the response headers names are converted to upper
+  cases and underscores (_). Also, if a header has only presented once, it
+  would not be wrapped inside an array. This is more consistent with
+  em-http-request, cool.io-http, and http_parser.rb
+
+* [`RestClient`] From now on, the default HTTP client, i.e. `RestClient` won't
+  follow any redirect. To follow redirect, please use `FollowRedirect`
+  middleware. Two reasons. One is that the underlying HTTP client should
+  be minimal. Another one is that a FollowRedirect middleware could be
+  used for all HTTP clients. This would make it more consistent across
+  all HTTP clients.
+
+* [`RestClient`] Added a patch to avoid `"123".to_i` returning `200`,
+  please see: <https://github.com/archiloque/rest-client/pull/103>
+  I would remove this once after this patch is merged.
+
+* [`RestClient`] Added a patch to properly returning response whenever
+  a redirect is happened. Please see:
+  <https://github.com/archiloque/rest-client/pull/118>
+  I would remove this once after this patch is merged.
+
+* [`FollowRedirect`] This middleware would follow the redirect. Pass
+  :max_redirects for the maximum redirect times. For example:
+
+  ``` ruby
+      Client = RestCore::Builder.client do
+        use FollowRedirect, 2 # default :max_redirects
+      end
+      client = Client.new
+      client.get('path', {}, :max_redirects => 5)
+  ```
+
+* [`Middleware`] Added `Middleware#run` which can return the underlying HTTP
+  client, if you need to know the underlying HTTP client can support
+  asynchronous requests or not.
+
+* [`Cache`] Now it's asynchrony-aware.
+* [`CommonLogger`] Now it's asynchrony-aware.
+* [`ErrorDetector`] Now it's asynchrony-aware.
+* [`ErrorHandler`] Now it's asynchrony-aware.
+* [`JsonDecode`] Now it's asynchrony-aware.
+* [`Timeout`] Now it's asynchrony-aware.
+
+* [`Universal`] `FollowRedirect` middleware is added.
+* [`Universal`] `Defaults` middleware is removed.
+
+* Added `RestCore::ASYNC` which should be the callback function which is
+  called whenever the response is available. It's similar to Rack's
+  async.callback.
+
+* Added `RestCore::TIMER` which is only used in Timeout middleware. We need
+  this to disable timer whenever the response is back.
+
+* [`EmHttpRequestAsync`] This HTTP client accepts a block to make asynchronous
+  HTTP requests via em-http-request gem.
+
+* [`EmHttpRequestFiber`] This HTTP client would make asynchronous HTTP
+  requests with em-http-request but also wrapped inside a fiber, so that it
+  looks synchronous to the program who calls it.
+
+* [`EmHttpRequest`] This HTTP client would would use `EmHttpRequestAsync` if
+  a block (`RestCore::ASYNC`) is passed, otherwise use `EmHttpRequestFiber`.
+
+* [`CoolioAsync`] This HTTP client is basically the same as
+  `EmHttpRequestAsync`, but using cool.io-http instead of em-http-request.
+
+* [`CoolioFiber`] This HTTP client is basically the same as
+  `EmHttpRequestFiber`, but using cool.io-http instead of em-http-request.
+
+* [`Coolio`] This HTTP client is basically the same as `EmHttpRequest`,
+    but using cool.io-http instead of em-http-request.
+
+* [`Auto`] This HTTP client would auto-select a suitable client. Under
+  eventmachine, it would use `EmHttpRequest`. Under cool.io, it would use
+  `Coolio`. Otherwise, it would use `RestClient`.
+
 ## rest-core 0.8.2 -- 2012-02-18
 
 ### Enhancement
 
-* [DefaultPayload] This middleware allows you to have default payload for
+* [`DefaultPayload`] This middleware allows you to have default payload for
   POST and PUT requests.
 
-* [Client] Now `lighten` would give all Unserializable to nil instead of false
+* [`Client`] Now `lighten` would give all Unserializable to nil instead of
+  false
 
 ### Bugs fixes
 
-* [ErrorDetector] Now it would do nothing instead of crashing if there's no
+* [`ErrorDetector`] Now it would do nothing instead of crashing if there's no
   any error_detector.
 
 ## rest-core 0.8.1 -- 2012-02-09
 
 ### Enhancement
 
-* [Wrapper] Introducing `Wrapper.default_app` (also `Builder.default_app`)
+* [`Wrapper`] Introducing `Wrapper.default_app` (also `Builder.default_app`)
   which you can change the default app from `RestClient` to other HTTP
   clients.
 
 ### Bugs fixes
 
-* [OAuth1Header] Correctly handle the signature when it comes to multipart
+* [`OAuth1Header`] Correctly handle the signature when it comes to multipart
   requests.
 
-* [ErrorDetectorHttp] Fixed argument error upon calling `lighten` for
+* [`ErrorDetectorHttp`] Fixed argument error upon calling `lighten` for
   clients using this middleware. (e.g. rest-more's Twitter client)
 
 ## rest-core 0.8.0 -- 2011-11-29
@@ -36,16 +146,16 @@ Changes are mostly related to OAuth.
 
 ### Incompatible changes
 
-* [OAuth1Header] `callback` is renamed to `oauth_callback`
-* [OAuth1Header] `verifier` is renamed to `oauth_verifier`
+* [`OAuth1Header`] `callback` is renamed to `oauth_callback`
+* [`OAuth1Header`] `verifier` is renamed to `oauth_verifier`
 
-* [Oauth2Header] The first argument is changed from `access_token` to
+* [`Oauth2Header`] The first argument is changed from `access_token` to
   `access_token_type`. Previously, the access_token_type is "OAuth" which
   is used in Mixi. But mostly, we might want to use "Bearer" (according to
   [OAuth 2.0 spec][]) Argument for the access_token is changed to the second
   argument.
 
-* [Defaults] Now we're no longer call `call` for any default values.
+* [`Defaults`] Now we're no longer call `call` for any default values.
   That is, if you're using this: `use s::Defaults, :data => lambda{{}}`
   that would break. Previously, this middleware would call `call` on the
   lambda so that `data` is default to a newly created hash. Now, it would
@@ -57,35 +167,36 @@ Changes are mostly related to OAuth.
 
 ### Enhancement
 
-* [AuthBasic] Added a new middleware which could do [basic authentication][].
+* [`AuthBasic`] Added a new middleware which could do
+  [basic authentication][].
 
-* [OAuth1Header] Introduced `data` which is a hash and is used to store
+* [`OAuth1Header`] Introduced `data` which is a hash and is used to store
   tokens and other information sent from authorization servers.
 
-* [ClientOauth1] Now `authorize_url!` accepts opts which you can pass
+* [`ClientOauth1`] Now `authorize_url!` accepts opts which you can pass
   `authorize_url!(:oauth_callback => 'http://localhost/callback')`.
 
-* [ClientOauth1] Introduced `authorize_url` which would not try to ask
+* [`ClientOauth1`] Introduced `authorize_url` which would not try to ask
   for a request token, instead, it would use the current token as the
   request token. If you don't understand what does this mean, then keep
   using `authorize_url!`, which would call this underneath.
 
-* [ClientOauth1] Introduced `authorized?`
-* [ClientOauth1] Now it would set `data['authorized'] = 'true'` when
+* [`ClientOauth1`] Introduced `authorized?`
+* [`ClientOauth1`] Now it would set `data['authorized'] = 'true'` when
   `authorize!` is called, and it is also used to check if we're authorized
   or not in `authorized?`
 
-* [ClientOauth1] Introduced `data_json` and `data_json=` which allow you to
+* [`ClientOauth1`] Introduced `data_json` and `data_json=` which allow you to
   serialize and deserialize `data` with JSON along with a `sig` to check
   if it hasn't been changed. You can put this into browser cookie. Because
   of the `sig`, you would know if the user changed something in data without
   using `consumer_secret` to generate a correct sig corresponded to the data.
 
-* [ClientOauth1] Introduced `oauth_token`, `oauth_token=`,
+* [`ClientOauth1`] Introduced `oauth_token`, `oauth_token=`,
   `oauth_token_secret`, `oauth_token_secret=`, `oauth_callback`,
   and `oauth_callback=` which take the advantage of `data`.
 
-* [ClientOauth1] Introduced `default_data` which is a hash.
+* [`ClientOauth1`] Introduced `default_data` which is a hash.
 
 [basic authentication]: http://en.wikipedia.org/wiki/Basic_access_authentication
 

data/Gemfile

@@ -3,6 +3,9 @@ source 'http://rubygems.org'
 
 gemspec
 
+gem 'rest-client'
+gem 'em-http-request'
+
 gem 'rake'
 gem 'bacon'
 gem 'rr'
@@ -16,6 +19,7 @@ gem 'ruby-hmac'
 
 platforms(:ruby) do
   gem 'yajl-ruby'
+  gem 'cool.io-http'
 end
 
 platforms(:jruby) do

data/README.md

@@ -1,9 +1,12 @@
-# rest-core [![Build Status](http://travis-ci.org/godfat/rest-core.png)](http://travis-ci.org/godfat/rest-core)
+# rest-core [![Build Status](https://secure.travis-ci.org/godfat/rest-core.png?branch=master)](http://travis-ci.org/godfat/rest-core)
 
 by Cardinal Blue <http://cardinalblue.com>
 
-Lin Jen-Shin ([godfat][]) had given a talk about rest-core on [RubyConf Taiwan 2011][talk].
-The slide is in English, but the talk is in Mandarin.
+Lin Jen-Shin ([godfat][]) had given a talk about rest-core on
+[RubyConf Taiwan 2011][talk]. The slide is in English, but the
+talk is in Mandarin.
+
+You can also read some other topics at [doc](https://github.com/cardinalblue/rest-core/blob/master/doc/ToC.md).
 
 [godfat]: https://github.com/godfat
 [talk]: http://rubyconf.tw/2011/#6
@@ -26,13 +29,30 @@ that allows you to build a REST client for any REST API. Or in the case of
 common APIs such as Facebook, Github, and Twitter, you can simply use the
 dedicated clients provided by [rest-more][].
 
-[rest-core]: http://github.com/cardinalblue/rest-core
-[rest-more]: http://github.com/cardinalblue/rest-more
+[rest-core]: https://github.com/cardinalblue/rest-core
+[rest-more]: https://github.com/cardinalblue/rest-more
+
+## FEATURES:
+
+* Modular interface for REST clients similar to WSGI/Rack for servers.
+* Asynchronous/Synchronous styles with or without fibers are both supported.
 
 ## REQUIREMENTS:
 
-* Tested with MRI (official CRuby) 1.8.7, 1.9.2, 1.9.3, Rubinius and JRuby
-* gem rest-client (for now)
+### Mandatory:
+
+* MRI (official CRuby) 1.8.7, 1.9.2, 1.9.3, Rubinius 1.8/1.9 and JRuby 1.8/1.9
+* gem rest-client
+
+### Optional:
+
+* Fibers only work on Ruby 1.9+
+* gem [em-http-request][] (if using eventmachine)
+* gem [cool.io-http][] (if using cool.io)
+* gem json or yajl-ruby (if using JsonDecode middleware)
+
+[em-http-request]: https://github.com/igrigorik/em-http-request
+[cool.io-http]: https://github.com/godfat/cool.io-http
 
 ## INSTALLATION:
 
@@ -40,8 +60,10 @@ dedicated clients provided by [rest-more][].
 
 Or if you want development version, put this in Gemfile:
 
+``` ruby
     gem 'rest-core', :git => 'git://github.com/cardinalblue/rest-core.git',
                      :submodules => true
+```
 
 If you just want to use Facebook or Twitter clients, please take a look at
 [rest-more][] which has a lot of clients built with rest-core.
@@ -50,6 +72,9 @@ If you just want to use Facebook or Twitter clients, please take a look at
 
 ## Build Your Own Clients:
 
+You can use `RestCore::Builder` to build your own dedicated client:
+
+``` ruby
     require 'rest-core'
 
     YourClient = RestCore::Builder.client do
@@ -57,31 +82,140 @@ If you just want to use Facebook or Twitter clients, please take a look at
       use s::DefaultSite , 'https://api.github.com/users/'
       use s::JsonDecode  , true
       use s::CommonLogger, method(:puts)
-      use s::Cache       , {}, 3600
-      run s::RestClient
+      use s::Cache       , nil, 3600
+      run s::RestClient # the simplest and easier HTTP client
     end
+```
 
-    client = YourClient.new
+And use it with per-instance basis (clients could have different
+configuration, e.g. different cache time or timeout time):
+
+``` ruby
+    client = YourClient.new(:cache => {})
     client.get('cardinalblue') # cache miss
     client.get('cardinalblue') # cache hit
 
     client.site = 'http://github.com/api/v2/json/user/show/'
     client.get('cardinalblue') # cache miss
     client.get('cardinalblue') # cache hit
+```
 
-Please see [rest-more][] for more complex examples,
-and [slides][] from [rubyconf.tw/2011][rubyconf.tw] for concepts.
+Runnable example is here: [example/rest-client.rb][]. Please see [rest-more][]
+for more complex examples, and [slides][] from [rubyconf.tw/2011][rubyconf.tw]
+for concepts.
 
+[example/rest-client.rb]: https://github.com/cardinalblue/rest-core/blob/master/example/rest-client.rb
 [rest-more]: https://github.com/cardinalblue/rest-more
 [rubyconf.tw]: http://rubyconf.tw/2011/#6
 [slides]: http://www.godfat.org/slide/2011-08-27-rest-core.html
 
+## Asynchronous HTTP Requests:
+
+I/O bound operations shouldn't be blocking the CPU! If you have a reactor,
+i.e. event loop, you should take the advantage of that to make HTTP requests
+non-blocking the whole process/thread. For now, we support eventmachine and
+cool.io. Below is an example for eventmachine:
+
+``` ruby
+    require 'rest-core'
+
+    AsynchronousClient = RestCore::Builder.client do
+      s = self.class # this is only for ruby 1.8!
+      use s::DefaultSite , 'https://api.github.com/users/'
+      use s::JsonDecode  , true
+      use s::CommonLogger, method(:puts)
+      use s::Cache       , nil, 3600
+      run s::EmHttpRequest
+    end
+```
+
+If you're passing a block, the block is called after the response is
+available. That is the block is the callback for the request.
+
+``` ruby
+    client = AsynchronousClient.new(:cache => {})
+    EM.run{
+      client.get('cardinalblue'){ |response|
+        p response
+        EM.stop
+      }
+    }
+```
+
+Otherwise, if you don't pass a block as the callback, EmHttpRequest (i.e.
+the HTTP client for eventmachine) would call `Fiber.yield` to yield to the
+original fiber, making asynchronous HTTP requests look like synchronous.
+If you don't understand what does this mean, you can take a look at
+[em-synchrony][]. It's basically the same idea.
+
+``` ruby
+    EM.run{
+      Fiber.new{
+        p client.get('cardinalblue')
+        EM.stop
+      }.resume
+    }
+```
+
+[em-synchrony]: https://github.com/igrigorik/em-synchrony
+
+Runnable example is here: [example/eventmachine.rb][].
+You can also make multi-requests synchronously like this:
+
+``` ruby
+    EM.run{
+      Fiber.new{
+        fiber = Fiber.current
+        result = {}
+        client.get('cardinalblue'){ |response|
+          result[0] = response
+          fiber.resume(result) if result.size == 2
+        }
+        client.get('cardinalblue'){ |response|
+          result[1] = response
+          fiber.resume(result) if result.size == 2
+        }
+        p Fiber.yield
+        EM.stop
+      }.resume
+    }
+```
+
+Runnable example is here: [example/multi.rb][].
+
+[example/eventmachine.rb]: https://github.com/cardinalblue/rest-core/blob/master/example/eventmachine.rb
+[example/multi.rb]: https://github.com/cardinalblue/rest-core/blob/master/example/multi.rb
+
+## Supported HTTP clients:
+
+* `RestCore::RestClient` (gem rest-client)
+* `RestCore::EmHttpRequest` (gem em-http-request)
+* `RestCore::Coolio` (gem cool.io)
+* `RestCore::Auto` (which would pick one of the above depending on the
+  context)
+
+## Build Your Own Middlewares:
+
+To be added.
+
+## Build Your Own HTTP clients:
+
+To be added.
+
 ## rest-core users:
 
 * [topcoder](https://github.com/miaout17/topcoder)
 * [s2sync](https://github.com/brucehsu/s2sync)
 * [s2sync_web](https://github.com/brucehsu/s2sync_web)
 
+## Powered sites:
+
+* [PicCollage](http://pic-collage.com/)
+
+## CHANGES:
+
+* [CHANGES](https://github.com/cardinalblue/rest-core/blob/master/CHANGES.md)
+
 ## GLOSSARY:
 
 * A _client_ is a class which can new connections to make requests.
@@ -184,7 +318,7 @@ and [slides][] from [rubyconf.tw/2011][rubyconf.tw] for concepts.
 
 Apache License 2.0
 
-Copyright (c) 2011, Cardinal Blue
+Copyright (c) 2011-2012, Cardinal Blue
 
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.

data/TODO.md

@@ -1,14 +1,4 @@
 # TODO
 
-## high
-
 * middleware revisit (how to initialize?)
-
-## medium
-
-* what about async request? yield callback? coroutine?
-* dependency?
-
-## low
-
 * test utility

data/doc/ToC.md

@@ -0,0 +1,7 @@
+
+# rest-core documentation
+
+## Table of Content
+
+* [Overview and Design Concept](design.md)
+* [Picking Dependency](dependency.md)

data/doc/dependency.md

@@ -0,0 +1,4 @@
+
+# Picking Dependency
+
+

data/doc/design.md

@@ -0,0 +1,4 @@
+
+# Overview and Design Concept
+
+

data/example/auto.rb

@@ -0,0 +1,51 @@
+
+require 'rest-core'
+require 'eventmachine'
+require 'cool.io'
+
+YourClient = RestCore::Builder.client do
+  s = self.class # this is only for ruby 1.8!
+  use s::DefaultSite , 'https://api.github.com/users/'
+  use s::JsonDecode  , true
+  use s::CommonLogger, method(:puts)
+  use s::Cache       , nil, 3600
+  run s::Auto
+end
+
+client = YourClient.new(:cache => {})
+p client.get('cardinalblue') # cache miss
+puts
+p client.get('cardinalblue') # cache hit
+
+puts
+
+client = YourClient.new(:cache => {})
+EM.run{
+  client.get('cardinalblue'){ |response|
+    p response
+    EM.stop
+  }
+}
+
+puts
+
+EM.run{
+  Fiber.new{
+    p client.get('cardinalblue')
+    EM.stop
+  }.resume
+}
+
+puts
+
+client = YourClient.new(:cache => {})
+Coolio::TimerWatcher.new(1).attach(Coolio::Loop.default).on_timer{detach}
+client.get('cardinalblue'){ |response|
+  p response
+}
+Coolio::Loop.default.run
+
+puts
+Coolio::TimerWatcher.new(1).attach(Coolio::Loop.default).on_timer{detach}
+Fiber.new{ p client.get('cardinalblue') }.resume
+Coolio::Loop.default.run

data/example/coolio.rb

@@ -0,0 +1,21 @@
+
+require 'rest-core'
+
+AsynchronousClient = RestCore::Builder.client do
+  s = self.class # this is only for ruby 1.8!
+  use s::DefaultSite , 'https://api.github.com/users/'
+  use s::JsonDecode  , true
+  use s::CommonLogger, method(:puts)
+  use s::Cache       , nil, 3600
+  run s::Coolio
+end
+
+AsynchronousClient.new.get('cardinalblue'){ |response|
+  p response
+}
+Coolio::Loop.default.run
+
+puts
+
+Fiber.new{ p AsynchronousClient.new.get('cardinalblue') }.resume
+Coolio::Loop.default.run

data/example/eventmachine.rb

@@ -0,0 +1,28 @@
+
+require 'rest-core'
+
+AsynchronousClient = RestCore::Builder.client do
+  s = self.class # this is only for ruby 1.8!
+  use s::DefaultSite , 'https://api.github.com/users/'
+  use s::JsonDecode  , true
+  use s::CommonLogger, method(:puts)
+  use s::Cache       , nil, 3600
+  run s::EmHttpRequest
+end
+
+client = AsynchronousClient.new(:cache => {})
+EM.run{
+  client.get('cardinalblue'){ |response|
+    p response
+    EM.stop
+  }
+}
+
+puts
+
+EM.run{
+  Fiber.new{
+    p client.get('cardinalblue')
+    EM.stop
+  }.resume
+}

data/example/multi.rb

@@ -0,0 +1,33 @@
+
+require 'rest-core'
+require 'eventmachine'
+RestCore::EmHttpRequest # there might be a autoload bug?
+                        # omitting this line would cause
+                        # stack level too deep (SystemStackError)
+
+YourClient = RestCore::Builder.client do
+  s = self.class # this is only for ruby 1.8!
+  use s::DefaultSite , 'https://api.github.com/users/'
+  use s::JsonDecode  , true
+  use s::CommonLogger, method(:puts)
+  use s::Cache       , nil, 3600
+  run s::Auto
+end
+
+client = YourClient.new
+EM.run{
+  Fiber.new{
+    fiber = Fiber.current
+    result = {}
+    client.get('cardinalblue'){ |response|
+      result[0] = response
+      fiber.resume(result) if result.size == 2
+    }
+    client.get('cardinalblue'){ |response|
+      result[1] = response
+      fiber.resume(result) if result.size == 2
+    }
+    p Fiber.yield
+    EM.stop
+  }.resume
+}

data/example/rest-client.rb

@@ -0,0 +1,16 @@
+
+require 'rest-core'
+
+YourClient = RestCore::Builder.client do
+  s = self.class # this is only for ruby 1.8!
+  use s::DefaultSite , 'https://api.github.com/users/'
+  use s::JsonDecode  , true
+  use s::CommonLogger, method(:puts)
+  use s::Cache       , nil, 3600
+  run s::RestClient
+end
+
+client = YourClient.new(:cache => {})
+p client.get('cardinalblue') # cache miss
+puts
+p client.get('cardinalblue') # cache hit

data/lib/rest-core/app/abstract/async_fiber.rb

@@ -0,0 +1,13 @@
+
+require 'rest-core/middleware'
+
+class RestCore::AsyncFiber
+  include RestCore::Middleware
+  def call env
+    if env[ASYNC]
+      async.call(env)
+    else
+      fiber.call(env)
+    end
+  end
+end