rest-core 1.0.3 → 2.0.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (87) hide show
  1. data/.travis.yml +6 -7
  2. data/CHANGES.md +137 -0
  3. data/Gemfile +1 -1
  4. data/README.md +183 -191
  5. data/TODO.md +5 -8
  6. data/example/multi.rb +31 -24
  7. data/example/simple.rb +28 -0
  8. data/example/use-cases.rb +194 -0
  9. data/lib/rest-core.rb +26 -19
  10. data/lib/rest-core/builder.rb +2 -2
  11. data/lib/rest-core/client.rb +40 -27
  12. data/lib/rest-core/client/universal.rb +16 -13
  13. data/lib/rest-core/client_oauth1.rb +5 -5
  14. data/lib/rest-core/engine/auto.rb +25 -0
  15. data/lib/rest-core/{app → engine}/dry.rb +1 -2
  16. data/lib/rest-core/engine/em-http-request.rb +39 -0
  17. data/lib/rest-core/engine/future/future.rb +106 -0
  18. data/lib/rest-core/engine/future/future_fiber.rb +39 -0
  19. data/lib/rest-core/engine/future/future_thread.rb +29 -0
  20. data/lib/rest-core/engine/rest-client.rb +56 -0
  21. data/lib/rest-core/middleware.rb +27 -5
  22. data/lib/rest-core/middleware/auth_basic.rb +5 -5
  23. data/lib/rest-core/middleware/bypass.rb +2 -2
  24. data/lib/rest-core/middleware/cache.rb +67 -54
  25. data/lib/rest-core/middleware/common_logger.rb +5 -8
  26. data/lib/rest-core/middleware/default_headers.rb +2 -2
  27. data/lib/rest-core/middleware/default_payload.rb +26 -2
  28. data/lib/rest-core/middleware/default_query.rb +4 -2
  29. data/lib/rest-core/middleware/default_site.rb +8 -6
  30. data/lib/rest-core/middleware/error_detector.rb +9 -16
  31. data/lib/rest-core/middleware/error_handler.rb +25 -11
  32. data/lib/rest-core/middleware/follow_redirect.rb +11 -14
  33. data/lib/rest-core/middleware/json_request.rb +19 -0
  34. data/lib/rest-core/middleware/json_response.rb +28 -0
  35. data/lib/rest-core/middleware/oauth1_header.rb +2 -7
  36. data/lib/rest-core/middleware/oauth2_header.rb +4 -7
  37. data/lib/rest-core/middleware/oauth2_query.rb +2 -2
  38. data/lib/rest-core/middleware/timeout.rb +21 -65
  39. data/lib/rest-core/middleware/timeout/{eventmachine_timer.rb → timer_em.rb} +3 -1
  40. data/lib/rest-core/middleware/timeout/timer_thread.rb +36 -0
  41. data/lib/rest-core/patch/multi_json.rb +8 -0
  42. data/lib/rest-core/test.rb +3 -12
  43. data/lib/rest-core/util/json.rb +65 -0
  44. data/lib/rest-core/util/parse_query.rb +2 -2
  45. data/lib/rest-core/version.rb +1 -1
  46. data/lib/rest-core/wrapper.rb +16 -16
  47. data/rest-core.gemspec +28 -27
  48. data/test/test_auth_basic.rb +14 -10
  49. data/test/test_builder.rb +7 -7
  50. data/test/test_cache.rb +126 -37
  51. data/test/test_client.rb +3 -1
  52. data/test/test_client_oauth1.rb +2 -3
  53. data/test/test_default_query.rb +17 -23
  54. data/test/test_em_http_request.rb +146 -0
  55. data/test/test_error_detector.rb +0 -1
  56. data/test/test_error_handler.rb +44 -0
  57. data/test/test_follow_redirect.rb +17 -19
  58. data/test/test_json_request.rb +28 -0
  59. data/test/test_json_response.rb +51 -0
  60. data/test/test_oauth1_header.rb +4 -4
  61. data/test/test_payload.rb +20 -12
  62. data/test/test_simple.rb +14 -0
  63. data/test/test_timeout.rb +11 -19
  64. data/test/test_universal.rb +5 -5
  65. data/test/test_wrapper.rb +19 -13
  66. metadata +28 -29
  67. data/doc/ToC.md +0 -7
  68. data/doc/dependency.md +0 -4
  69. data/doc/design.md +0 -4
  70. data/example/auto.rb +0 -51
  71. data/example/coolio.rb +0 -21
  72. data/example/eventmachine.rb +0 -30
  73. data/example/rest-client.rb +0 -16
  74. data/lib/rest-core/app/abstract/async_fiber.rb +0 -13
  75. data/lib/rest-core/app/auto.rb +0 -23
  76. data/lib/rest-core/app/coolio-async.rb +0 -32
  77. data/lib/rest-core/app/coolio-fiber.rb +0 -30
  78. data/lib/rest-core/app/coolio.rb +0 -9
  79. data/lib/rest-core/app/em-http-request-async.rb +0 -37
  80. data/lib/rest-core/app/em-http-request-fiber.rb +0 -45
  81. data/lib/rest-core/app/em-http-request.rb +0 -9
  82. data/lib/rest-core/app/rest-client.rb +0 -41
  83. data/lib/rest-core/middleware/json_decode.rb +0 -93
  84. data/lib/rest-core/middleware/timeout/coolio_timer.rb +0 -10
  85. data/pending/test_multi.rb +0 -123
  86. data/pending/test_test_util.rb +0 -86
  87. data/test/test_json_decode.rb +0 -24
@@ -1,11 +1,10 @@
1
1
  before_install: 'git submodule update --init'
2
- script: 'bundle exec rake test'
2
+ script: 'ruby -r bundler/setup -S rake test'
3
+
4
+ env:
5
+ - 'RBXOPT=-X19'
3
6
 
4
7
  rvm:
5
- - 1.8.7
6
- - 1.9.2
7
8
  - 1.9.3
8
- - rbx-18mode
9
- - rbx-19mode
10
- - jruby-18mode
11
- - jruby-19mode
9
+ - rbx-head
10
+ - jruby-head
data/CHANGES.md CHANGED
@@ -1,5 +1,142 @@
1
1
  # CHANGES
2
2
 
3
+ ## rest-core 2.0.0
4
+
5
+ This is a major release which introduces some incompatible changes.
6
+ This is intended to cleanup some internal implementation and introduce
7
+ a new mechanism to handle multiple requests concurrently, avoiding needless
8
+ block.
9
+
10
+ Before we go into detail, here's who can upgrade without changing anything,
11
+ and who should make a few adjustments in their code:
12
+
13
+ * If you're only using rest-more, e.g. `RC::Facebook` or `RC::Twitter`, etc.,
14
+ you don't have to change anything. This won't affect rest-more users.
15
+ (except that JsonDecode is renamed to JsonResponse, and json_decode is
16
+ renamed to json_response.)
17
+
18
+ * If you're only using rest-core's built in middlewares to build your own
19
+ clients, you don't have to change anything as well. All the hard works are
20
+ done in rest-core. (except that ErrorHandler works a bit differently now.
21
+ We'll talk about detail later.)
22
+
23
+ * If you're building your own middlewares, then you are the ones who need to
24
+ make changes. `RC::ASYNC` is changed to a flag to mean whether the callback
25
+ should be called directly, or only after resuming from the future (fiber
26
+ or thread). And now you have always to get the response from `yield`, that
27
+ is, you're forced to pass a callback to `call`.
28
+
29
+ This might be a bit user unfriendly at first glimpse, but it would much
30
+ simplify the internal structure of rest-core, because in the middlewares,
31
+ you don't have to worry if the user would pass a callback or not, branching
32
+ everywhere to make it work both synchronously and asynchronously.
33
+
34
+ Also, the old fiber based asynchronous HTTP client is removed, in favor
35
+ of the new _future_ based approach. The new one is more like a superset
36
+ of the old one, which have anything the old one can provide. Yet internally
37
+ it works a lot differently. They are both synchronous to the outsides,
38
+ but while the old one is also synchronous inside, the new one is
39
+ asynchronous inside, just like the purely asynchronous HTTP client.
40
+
41
+ That is, internally, it's always asynchronously, and fiber/async didn't
42
+ make much difference here now. This is also the reason why I removed
43
+ the old fiber one. This would make the middleware implementation much
44
+ easier, considering much fewer possible cases.
45
+
46
+ If you don't really understand what above does mean, then just remember,
47
+ now we ask all middlewares work asynchronously. You have always to work
48
+ with callbacks which passed along in `app.call(env){ |response| }`
49
+ That's it.
50
+
51
+ So what's the most important improvement? From now on, we have only two
52
+ modes. One is callback mode, in which case `env[ASYNC]` would be set, and
53
+ the callback would be called. No exception would be raised in this case.
54
+ If there's an exception, then it would be passed to the block instead.
55
+
56
+ The other mode, which is synchronous, is achieved by the futures. We have
57
+ two different kinds of futures for now, one is thread based, and the other
58
+ is fiber based. For RestClient, thread based future would be used. For
59
+ EventMachine, depending on the context, if the future is created on the
60
+ main thread, then it would assume it's also wrapped inside a fiber. Since,
61
+ we can never block the event loop! If you're not calling it in a thread,
62
+ you must call it in a fiber. But if you're calling it in a thread, then
63
+ the thread based future would be picked. This is because otherwise it won't
64
+ work well exchanging information around threads and fibers.
65
+
66
+ In short, rest-core would run concurrently in all contexts, archived by
67
+ either threads or fibers depending on the context, and it would pick the
68
+ right strategy for you.
69
+
70
+ You can see [use-cases.rb][] for all possible use cases.
71
+
72
+ It's a bit outdated, but you can also checkout my blog post.
73
+ [rest-core 2.0 roadmap, thunk based response][post]
74
+ (p.s. now thunk is renamed to future)
75
+
76
+ [use-cases.rb]: https://github.com/cardinalblue/rest-core/blob/master/example/use-cases.rb
77
+ [post]: http://blogger.godfat.org/2012/06/rest-core-20-roadmap-thunk-based.html
78
+
79
+ ### Incompatible changes
80
+
81
+ * [JsonDecode] is renamed to JsonResponse, and json_decode is also renamed
82
+ to json_response.
83
+ * [Json] Now you can use `Json.decode` and `Json.encode` to parse and
84
+ generate JSONs, instead of `JsonDecode.json_decode`.
85
+ * [Cache] Support for "cache.post" is removed.
86
+ * [Cache] The cache key is changed accordingly to support cache for headers
87
+ and HTTP status. If you don't have persistent cache, this doesn't matter.
88
+
89
+ * [EmHttpRequestFiber] is removed in favor of `EmHttpRequest`
90
+ * cool.io support is removed.
91
+ * You must provide a block to `app.call(env){ ... }`.
92
+ * Rename Wrapper#default_app to Wrapper#default_engine
93
+
94
+ ### Enhancement
95
+
96
+ * The default engine is changed from `RestClient` to `Auto`, which would
97
+ be using `EmHttpRequest` under the context of a event loop, while
98
+ use `RestClient` in other context as before.
99
+
100
+ * `RestCore.eagerload` is introduced to load all constants eagerly. You can
101
+ use this before loading the application to avoid thread-safety issue in
102
+ autoload. For the lazies.
103
+
104
+ * [JsonResponse] This is originally JsonDecode, and now we prefer multi_json
105
+ first, yajl-ruby second, lastly json.
106
+ * [JsonResponse] give JsonResponse a default header Accept: application/json,
107
+ thanks @ayamomiji
108
+ * [JsonRequest] This middleware would encode your payload into a JSON.
109
+ * [CommonLogger] Now we log the request method as well.
110
+ * [DefaultPayload] Accept arbitrary payload.
111
+ * [DefaultQuery] Now before merging queries, converting every single key into
112
+ a string. This allows you to use :symbols for default query.
113
+
114
+ * [ErrorHandler] So now ErrorHandler is working differently. It would first
115
+ try to see if `env[FAIL]` has any exception in it. If there is, then raise
116
+ it. Otherwise it would call error_handler and expect it to generate an
117
+ error object. If the error object is an exception, then raise it. If it's
118
+ not, then it would merge it into `env[FAIL]`. On the other hand, in the
119
+ case of using callbacks instead of futures, it would pass the exception
120
+ as the `env[RESPONSE_BODY]` instead. The reason is that you can't raise
121
+ an exception asynchronously and handle it safely.
122
+
123
+ * [Cache] Now response headers and HTTP status are also cached.
124
+ * [Cache] Not only GET requests are cached, HEAD and OPTIONS are cached too.
125
+ * [Cache] The cache key is also respecting the request headers too. Suppose
126
+ you're making a request with different Accept header.
127
+
128
+ * [Client] Add Client#wait which would block until all requests for this
129
+ particular client are done.
130
+
131
+ ### Bugs fixes
132
+
133
+ * [Middleware] Sort the query before generating the request URI, making
134
+ sure the order is always the same.
135
+ * [Middleware] The middleware could have no members at all.
136
+ * [ParseQuery] The fallback function for the absence of Rack is fixed.
137
+ * [Auto] Only use EmHttpRequest if em-http-request is loaded,
138
+ thanks @ayamomiji
139
+
3
140
  ## rest-core 1.0.3 -- 2012-08-15
4
141
 
5
142
  ### Enhancement
data/Gemfile CHANGED
@@ -13,13 +13,13 @@ gem 'webmock'
13
13
 
14
14
  gem 'json'
15
15
  gem 'json_pure'
16
+ gem 'multi_json'
16
17
 
17
18
  gem 'rack'
18
19
  gem 'ruby-hmac'
19
20
 
20
21
  platforms(:ruby) do
21
22
  gem 'yajl-ruby'
22
- gem 'cool.io-http'
23
23
  end
24
24
 
25
25
  platforms(:jruby) do
data/README.md CHANGED
@@ -6,8 +6,6 @@ Lin Jen-Shin ([godfat][]) had given a talk about rest-core on
6
6
  [RubyConf Taiwan 2011][talk]. The slide is in English, but the
7
7
  talk is in Mandarin.
8
8
 
9
- You can also read some other topics at [doc](https://github.com/cardinalblue/rest-core/blob/master/doc/ToC.md).
10
-
11
9
  [godfat]: https://github.com/godfat
12
10
  [talk]: http://rubyconf.tw/2011/#6
13
11
 
@@ -24,47 +22,45 @@ Modular Ruby clients interface for REST APIs
24
22
 
25
23
  There has been an explosion in the number of REST APIs available today.
26
24
  To address the need for a way to access these APIs easily and elegantly,
27
- we have developed [rest-core][], which consists of composable middleware
25
+ we have developed rest-core, which consists of composable middleware
28
26
  that allows you to build a REST client for any REST API. Or in the case of
29
27
  common APIs such as Facebook, Github, and Twitter, you can simply use the
30
28
  dedicated clients provided by [rest-more][].
31
29
 
32
- [rest-core]: https://github.com/cardinalblue/rest-core
33
30
  [rest-more]: https://github.com/cardinalblue/rest-more
34
31
 
35
32
  ## FEATURES:
36
33
 
37
34
  * Modular interface for REST clients similar to WSGI/Rack for servers.
38
- * Asynchronous/Synchronous styles with or without fibers are both supported.
35
+ * Concurrent requests with synchronous or asynchronous interfaces with
36
+ fibers or threads are both supported.
39
37
 
40
38
  ## REQUIREMENTS:
41
39
 
42
40
  ### Mandatory:
43
41
 
44
- * MRI (official CRuby) 1.8.7, 1.9.2, 1.9.3, Rubinius 1.8/1.9 and JRuby 1.8/1.9
42
+ * MRI (official CRuby) 1.9.3, Rubinius 1.9 and JRuby 1.9
45
43
  * gem rest-client
46
44
 
47
45
  ### Optional:
48
46
 
49
- * Fibers only work on Ruby 1.9+
50
47
  * gem [em-http-request][] (if using eventmachine)
51
- * gem [cool.io-http][] (if using cool.io)
52
- * gem json or yajl-ruby (if using `JsonDecode` middleware)
48
+ * gem json or yajl-ruby, or multi_json (if `JsonResponse` or
49
+ `JsonRequest` middlewares are used)
53
50
 
54
51
  [em-http-request]: https://github.com/igrigorik/em-http-request
55
- [cool.io-http]: https://github.com/godfat/cool.io-http
56
52
 
57
53
  ## INSTALLATION:
58
54
 
59
55
  ``` shell
60
- gem install rest-core
56
+ gem install rest-core
61
57
  ```
62
58
 
63
59
  Or if you want development version, put this in Gemfile:
64
60
 
65
61
  ``` ruby
66
- gem 'rest-core', :git => 'git://github.com/cardinalblue/rest-core.git',
67
- :submodules => true
62
+ gem 'rest-core', :git => 'git://github.com/cardinalblue/rest-core.git',
63
+ :submodules => true
68
64
  ```
69
65
 
70
66
  If you just want to use Facebook or Twitter clients, please take a look at
@@ -74,235 +70,231 @@ If you just want to use Facebook or Twitter clients, please take a look at
74
70
 
75
71
  ## Build Your Own Clients:
76
72
 
77
- You can use `RestCore::Builder` to build your own dedicated client:
73
+ You can use `RestCore::Builder` to build your own dedicated clients.
74
+ Note that `RC` is an alias of `RestCore`
78
75
 
79
76
  ``` ruby
80
- require 'rest-core'
81
-
82
- YourClient = RestCore::Builder.client do
83
- s = RestCore
84
- use s::DefaultSite , 'https://api.github.com/users/'
85
- use s::JsonDecode , true
86
- use s::CommonLogger, method(:puts)
87
- use s::Cache , nil, 3600
88
- run s::RestClient # the simplest and easier HTTP client
89
- end
77
+ require 'rest-core'
78
+ YourClient = RC::Builder.client do
79
+ use RC::DefaultSite , 'https://api.github.com/users/'
80
+ use RC::JsonResponse, true
81
+ use RC::CommonLogger, method(:puts)
82
+ use RC::Cache , nil, 3600
83
+ end
90
84
  ```
91
85
 
92
86
  And use it with per-instance basis (clients could have different
93
87
  configuration, e.g. different cache time or timeout time):
94
88
 
95
89
  ``` ruby
96
- client = YourClient.new(:cache => {})
97
- client.get('cardinalblue') # cache miss
98
- client.get('cardinalblue') # cache hit
90
+ client = YourClient.new(:cache => {})
91
+ client.get('cardinalblue') # cache miss
92
+ client.get('cardinalblue') # cache hit
99
93
 
100
- client.site = 'http://github.com/api/v2/json/user/show/'
101
- client.get('cardinalblue') # cache miss
102
- client.get('cardinalblue') # cache hit
94
+ client.site = 'http://github.com/api/v2/json/user/show/'
95
+ client.get('cardinalblue') # cache miss
96
+ client.get('cardinalblue') # cache hit
103
97
  ```
104
98
 
105
- Runnable example is here: [example/rest-client.rb][]. Please see [rest-more][]
106
- for more complex examples, and [slides][] from [rubyconf.tw/2011][rubyconf.tw]
107
- for concepts.
108
-
109
- [example/rest-client.rb]: https://github.com/cardinalblue/rest-core/blob/master/example/rest-client.rb
110
- [rest-more]: https://github.com/cardinalblue/rest-more
111
- [slides]: http://www.godfat.org/slide/2011-08-27-rest-core.html
112
- [rubyconf.tw]: http://rubyconf.tw/2011/#6
113
-
114
- ## Asynchronous HTTP Requests:
115
-
116
- I/O bound operations shouldn't be blocking the CPU! If you have a reactor,
117
- i.e. event loop, you should take the advantage of that to make HTTP requests
118
- not block the whole process/thread. For now, we support eventmachine and
119
- cool.io. Below is an example for eventmachine:
99
+ You can also make concurrent requests easily:
100
+ (see "Advanced Concurrent HTTP Requests -- Embrace the Future" for detail)
120
101
 
121
102
  ``` ruby
122
- require 'rest-core'
123
-
124
- AsynchronousClient = RestCore::Builder.client do
125
- s = RestCore
126
- use s::DefaultSite , 'https://api.github.com/users/'
127
- use s::JsonDecode , true
128
- use s::CommonLogger, method(:puts)
129
- use s::Cache , nil, 3600
130
- run s::EmHttpRequest
131
- end
103
+ a = [client.get('cardinalblue')['name'], client.get('godfat')['name']]
104
+ puts "It's not blocking... but doing concurrent requests underneath"
105
+ p a # here we want the values, so it blocks here
106
+ puts "DONE"
132
107
  ```
133
108
 
134
- If you're passing a block, the block is called after the response is
135
- available. That is the block is the callback for the request.
109
+ Callback mode also available:
136
110
 
137
111
  ``` ruby
138
- client = AsynchronousClient.new
139
- EM.run{
140
- client.get('cardinalblue'){ |response|
141
- p response
142
- EM.stop
143
- }
144
- puts "It's not blocking..."
145
- }
112
+ client.get('cardinalblue'){ |v| p v }
113
+ puts "It's not blocking... but doing concurrent requests underneath"
114
+ client.wait # we block here to wait for the request done
115
+ puts "DONE"
146
116
  ```
147
117
 
148
- Otherwise, if you don't pass a block as the callback, EmHttpRequest (i.e.
149
- the HTTP client for eventmachine) would call `Fiber.yield` to yield to the
150
- original fiber, making asynchronous HTTP requests look like synchronous.
151
- If you don't understand what does this mean, you can take a look at
152
- [em-synchrony][]. It's basically the same idea.
118
+ Runnable example is at: [example/simple.rb][]. Please see [rest-more][]
119
+ for more complex examples to build clients, and [slides][] from
120
+ [rubyconf.tw/2011][rubyconf.tw] for concepts.
153
121
 
154
- ``` ruby
155
- EM.run{
156
- Fiber.new{
157
- p client.get('cardinalblue')
158
- EM.stop
159
- }.resume
160
- puts "It's not blocking..."
161
- }
162
- ```
163
-
164
- [em-synchrony]: https://github.com/igrigorik/em-synchrony
165
-
166
- Runnable example is here: [example/eventmachine.rb][].
167
- You can also make multi-requests synchronously like this:
168
-
169
- ``` ruby
170
- EM.run{
171
- Fiber.new{
172
- fiber = Fiber.current
173
- result = {}
174
- client.get('cardinalblue'){ |response|
175
- result[0] = response
176
- fiber.resume(result) if result.size == 2
177
- }
178
- client.get('cardinalblue'){ |response|
179
- result[1] = response
180
- fiber.resume(result) if result.size == 2
181
- }
182
- p Fiber.yield
183
- EM.stop
184
- }.resume
185
- puts "It's not blocking..."
186
- }
187
- ```
188
-
189
- Runnable example is here: [example/multi.rb][].
190
-
191
- [example/eventmachine.rb]: https://github.com/cardinalblue/rest-core/blob/master/example/eventmachine.rb
192
- [example/multi.rb]: https://github.com/cardinalblue/rest-core/blob/master/example/multi.rb
193
-
194
- ## Supported HTTP clients:
122
+ [example/simple.rb]: https://github.com/cardinalblue/rest-core/blob/master/example/simple.rb
123
+ [rest-more]: https://github.com/cardinalblue/rest-more
124
+ [slides]: http://www.godfat.org/slide/2011-08-27-rest-core.html
125
+ [rubyconf.tw]: http://rubyconf.tw/2011/#6
195
126
 
196
- * `RestCore::RestClient` (gem rest-client)
197
- * `RestCore::EmHttpRequest` (gem em-http-request)
198
- * `RestCore::Coolio` (gem cool.io)
199
- * `RestCore::Auto` (which would pick one of the above depending on the
200
- context)
127
+ ## List of built-in Middlewares:
128
+
129
+ * `RC::AuthBasic`
130
+ * `RC::Bypass`
131
+ * `RC::Cache`
132
+ * `RC::CommonLogger`
133
+ * `RC::DefaultHeaders`
134
+ * `RC::DefaultPayload`
135
+ * `RC::DefaultQuery`
136
+ * `RC::DefaultSite`
137
+ * `RC::Defaults`
138
+ * `RC::ErrorDetector`
139
+ * `RC::ErrorDetectorHttp`
140
+ * `RC::ErrorHandler`
141
+ * `RC::FollowRedirect`
142
+ * `RC::JsonRequest`
143
+ * `RC::JsonResponse`
144
+ * `RC::Oauth1Header`
145
+ * `RC::Oauth2Header`
146
+ * `RC::Oauth2Query`
147
+ * `RC::Timeout`
201
148
 
202
149
  ## Build Your Own Middlewares:
203
150
 
204
151
  To be added.
205
152
 
206
- ## Build Your Own HTTP clients:
207
-
208
- To be added.
209
-
210
- ## rest-core users:
211
-
212
- * [topcoder](https://github.com/miaout17/topcoder)
213
- * [s2sync](https://github.com/brucehsu/s2sync)
214
- * [s2sync_web](https://github.com/brucehsu/s2sync_web)
153
+ ## Advanced Concurrent HTTP Requests -- Embrace the Future
215
154
 
216
- ## Powered sites:
155
+ ### The Interface
217
156
 
218
- * [PicCollage](http://pic-collage.com/)
157
+ There are a number of different ways to make concurrent requests in
158
+ rest-core. They could be roughly categorized to two different forms.
159
+ One is using the well known callbacks, while the other one is using
160
+ through a technique called [future][]. Basically, it means it would
161
+ return you a promise, which would eventually become the real value
162
+ (response here) you were asking for whenever you really want it.
163
+ Otherwise, the program keeps running until the value is evaluated,
164
+ and blocks there if the computation (response) hasn't been done yet.
165
+ If the computation is already done, then it would simply return you
166
+ the result.
219
167
 
220
- ## CHANGES:
168
+ Here's a very simple example for using futures:
221
169
 
222
- * [CHANGES](https://github.com/cardinalblue/rest-core/blob/master/CHANGES.md)
170
+ ``` ruby
171
+ require 'rest-core'
172
+ YourClient = RC::Builder.client do
173
+ use RC::DefaultSite , 'https://api.github.com/users/'
174
+ use RC::JsonResponse, true
175
+ use RC::CommonLogger, method(:puts)
176
+ end
177
+
178
+ client = YourClient.new
179
+ puts "rest-client with threads doing concurrent requests"
180
+ a = [client.get('cardinalblue')['name'], client.get('godfat')['name']]
181
+ puts "It's not blocking... but doing concurrent requests underneath"
182
+ p a # here we want the values, so it blocks here
183
+ puts "DONE"
184
+ ```
223
185
 
224
- ## GLOSSARY:
186
+ And here's a corresponded version for using callbacks:
225
187
 
226
- * A _client_ is a class which can new connections to make requests.
227
- For instance, `RestCore::Facebook.new.get('4')`
188
+ ``` ruby
189
+ require 'rest-core'
190
+ YourClient = RC::Builder.client do
191
+ use RC::DefaultSite , 'https://api.github.com/users/'
192
+ use RC::JsonResponse, true
193
+ use RC::CommonLogger, method(:puts)
194
+ end
195
+
196
+ client = YourClient.new
197
+ puts "rest-client with threads doing concurrent requests"
198
+ client.get('cardinalblue'){ |v|
199
+ p v['name']
200
+ }.
201
+ get('godfat'){ |v|
202
+ p v['name']
203
+ }
204
+ puts "It's not blocking... but doing concurrent requests underneath"
205
+ client.wait # until all requests are done
206
+ puts "DONE"
207
+ ```
228
208
 
229
- * An _app_ is an HTTP client which would do the underneath HTTP requests.
230
- For instance, `RestCore::RestClient` is an HTTP client which uses
231
- rest-client gem (`::RestClient`) to make HTTP requests.
209
+ You can pick whatever works for you.
232
210
 
233
- * A _middleware_ is a component for a rest-core stack.
234
- For instance, `RestCore::DefaultSite` is a middleware which would add
235
- default site URL in front of the request URI if it is not started with
236
- http://, thus you can do this: `RestCore::Facebook.get('4')` without
237
- specifying where the site (Facebook) it is.
211
+ [future]: http://en.wikipedia.org/wiki/Futures_and_promises
238
212
 
239
- * `RestCore::Wrapper` is a utility which could help you wrap a number of
240
- middlewares into another middleware. Currently, it's used in
241
- `RestCore::Buidler` and `RestCore::Cache`.
213
+ ### What Concurrency Model to Choose?
242
214
 
243
- * `RestCore::Builder` is a utility which could help you build a _client_
244
- with a collection of _middlewares_ and an _app_. i.e. a rest-core stack.
215
+ In the above example, we're using rest-client with threads, which works
216
+ for most of cases. But you might also want to use em-http-request with
217
+ EventMachine, which is using a faster HTTP parser. In theory, it should
218
+ be much more efficient than rest-client and threads.
245
219
 
246
- * `RestCore::Middleware` is a utility which could help you build a non-trivial
247
- middleware. More explanation to come...
220
+ To pick em-http-request, you must run the requests inside the EventMachine's
221
+ event loop, and also wrap your request with either a thread or a fiber,
222
+ because we can't block the event loop and ask em-http-request to finish
223
+ its job making requests.
248
224
 
249
- * `RestCore::Client` is a module which would be included in a generated
250
- _client_ by `RestCore::Builder`. It contains a number of convenient
251
- functions which is generally useful.
225
+ Here's an example of using em-http-request with threads:
252
226
 
253
- * `RestCore::ClientOAuth1` is a module which should be included in a OAuth1.0
254
- client. It contains a number of convenient functions which is useful for an
255
- OAuth 1.0 client.
227
+ ``` ruby
228
+ require 'em-http-request'
229
+ require 'rest-core'
230
+ YourClient = RC::Builder.client do
231
+ use RC::DefaultSite , 'https://api.github.com/users/'
232
+ use RC::JsonResponse, true
233
+ use RC::CommonLogger, method(:puts)
234
+ end
235
+
236
+ client = YourClient.new
237
+ puts "eventmachine with threads doing concurrent requests"
238
+ EM.run{
239
+ Thread.new{
240
+ p [client.get('cardinalblue')['name'], client.get('godfat')['name']]
241
+ puts "DONE"
242
+ EM.stop
243
+ }
244
+ puts "It's not blocking... but doing concurrent requests underneath"
245
+ }
246
+ ```
256
247
 
257
- * An `env` is a hash which contains all the information for both request and
258
- response. It's mostly seen in `@app.call(env)` See other explanation
259
- such as `env[RestCore::REQUEST_METHOD]` for more detail.
248
+ And here's an example of using em-http-request with fibers:
260
249
 
261
- * `env[RestCore::REQUEST_METHOD]` is a symbol representing which HTTP method
262
- would be used in the subsequent HTTP request. The possible values are
263
- either: `:get`, `:post`, `:put` or `:delete`.
250
+ ``` ruby
251
+ require 'fiber' # remember to require fiber first,
252
+ require 'em-http-request' # or rest-core won't pick fibers
253
+ require 'rest-core'
254
+ YourClient = RC::Builder.client do
255
+ use RC::DefaultSite , 'https://api.github.com/users/'
256
+ use RC::JsonResponse, true
257
+ use RC::CommonLogger, method(:puts)
258
+ end
259
+
260
+ client = YourClient.new
261
+ puts "eventmachine with fibers doing concurrent requests"
262
+ EM.run{
263
+ Fiber.new{
264
+ p [client.get('cardinalblue')['name'], client.get('godfat')['name']]
265
+ puts "DONE"
266
+ EM.stop
267
+ }
268
+ puts "It's not blocking... but doing concurrent requests underneath"
269
+ }
270
+ ```
264
271
 
265
- * `env[RestCore::REQUEST_PATH]` is a string representing which HTTP path
266
- would be used in the subsequent HTTP request. This path could also include
267
- the protocol, not only the path. e.g. `"http://graph.facebook.com/4"` or
268
- simply `"4"`. In the case of built-in Facebook client, the
269
- `RestCore::DefaultSite` middleware would take care of the site.
272
+ As you can see, both of them are quite similar to each other, because the
273
+ idea behind the scene is the same. If you don't know what concurrency model
274
+ to pick, start with rest-client since it's the easiest one to setup.
270
275
 
271
- * `env[RestCore::REQUEST_QUERY]` is a hash which keys are query keys and
272
- values are query values. Both keys and values' type should be String, not
273
- Symbol. Values with nil or false would be ignored. Both keys and values
274
- would be escaped automatically.
276
+ A full runnable example is at: [example/multi.rb][]. If you want to know
277
+ all the possible use cases, you can also see: [example/use-cases.rb][]. It's
278
+ also served as a test for each possible combinations, so it's quite complex
279
+ and complete.
275
280
 
276
- * `env[RestCore::REQUEST_PAYLOAD]` is a hash which keys are payload keys and
277
- values are payload values. Both keys and values' type should be String,
278
- not Symbol. Values with nil or false would be ignored. Both keys and values
279
- would be escaped automatically.
281
+ [example/multi.rb]: https://github.com/cardinalblue/rest-core/blob/master/example/multi.rb
280
282
 
281
- * `env[RestCore::REQUEST_HEADERS]` is a hash which keys are header names and
282
- values are header values. Both keys and values' type should be String,
283
- not Symbol. Values with nil or false would be ignored.
283
+ [example/use-cases.rb]: https://github.com/cardinalblue/rest-core/blob/master/example/use-cases.rb
284
284
 
285
- * `env[RestCore::RESPONSE_BODY]` is a string which is returned by the server.
286
- Might be nil if there's no response or not yet making HTTP request.
285
+ ## rest-core users:
287
286
 
288
- * `env[RestCore::RESPONSE_STATUS]` is a number which is returned by the
289
- server for the HTTP status. Might be nil if there's no response or not
290
- yet making HTTP request.
287
+ * [topcoder](https://github.com/miaout17/topcoder)
288
+ * [s2sync](https://github.com/brucehsu/s2sync)
289
+ * [s2sync_web](https://github.com/brucehsu/s2sync_web)
291
290
 
292
- * `env[RestCore::RESPONSE_HEADERS]` is a hash which is returned by the server
293
- for the response headers. Both keys and values' type should be String.
291
+ ## Powered sites:
294
292
 
295
- * `env[RestCore::DRY]` is a boolean (either `true` or `false` or `nil`) which
296
- indicates that if we're only asking for modified `env`, instead of making
297
- real requests. It's used to ask for the real request URI, etc.
293
+ * [PicCollage](http://pic-collage.com/)
298
294
 
299
- * `env[RestCore::FAIL]` is an array which contains failing events. Events
300
- could be any objects, it's handled by `RestCore::ErrorDetector` or any
301
- other custom _middleware_.
295
+ ## CHANGES:
302
296
 
303
- * `env[RestCore::LOG]` is an array which contains logging events. Events
304
- could be any objects, it's handled by `RestCore::CommonLogger` or
305
- any other custom _middleware_.
297
+ * [CHANGES](https://github.com/cardinalblue/rest-core/blob/master/CHANGES.md)
306
298
 
307
299
  ## CONTRIBUTORS:
308
300