rest-core 1.0.3 → 2.0.0
Sign up to get free protection for your applications and to get access to all the features.
- data/.travis.yml +6 -7
- data/CHANGES.md +137 -0
- data/Gemfile +1 -1
- data/README.md +183 -191
- data/TODO.md +5 -8
- data/example/multi.rb +31 -24
- data/example/simple.rb +28 -0
- data/example/use-cases.rb +194 -0
- data/lib/rest-core.rb +26 -19
- data/lib/rest-core/builder.rb +2 -2
- data/lib/rest-core/client.rb +40 -27
- data/lib/rest-core/client/universal.rb +16 -13
- data/lib/rest-core/client_oauth1.rb +5 -5
- data/lib/rest-core/engine/auto.rb +25 -0
- data/lib/rest-core/{app → engine}/dry.rb +1 -2
- data/lib/rest-core/engine/em-http-request.rb +39 -0
- data/lib/rest-core/engine/future/future.rb +106 -0
- data/lib/rest-core/engine/future/future_fiber.rb +39 -0
- data/lib/rest-core/engine/future/future_thread.rb +29 -0
- data/lib/rest-core/engine/rest-client.rb +56 -0
- data/lib/rest-core/middleware.rb +27 -5
- data/lib/rest-core/middleware/auth_basic.rb +5 -5
- data/lib/rest-core/middleware/bypass.rb +2 -2
- data/lib/rest-core/middleware/cache.rb +67 -54
- data/lib/rest-core/middleware/common_logger.rb +5 -8
- data/lib/rest-core/middleware/default_headers.rb +2 -2
- data/lib/rest-core/middleware/default_payload.rb +26 -2
- data/lib/rest-core/middleware/default_query.rb +4 -2
- data/lib/rest-core/middleware/default_site.rb +8 -6
- data/lib/rest-core/middleware/error_detector.rb +9 -16
- data/lib/rest-core/middleware/error_handler.rb +25 -11
- data/lib/rest-core/middleware/follow_redirect.rb +11 -14
- data/lib/rest-core/middleware/json_request.rb +19 -0
- data/lib/rest-core/middleware/json_response.rb +28 -0
- data/lib/rest-core/middleware/oauth1_header.rb +2 -7
- data/lib/rest-core/middleware/oauth2_header.rb +4 -7
- data/lib/rest-core/middleware/oauth2_query.rb +2 -2
- data/lib/rest-core/middleware/timeout.rb +21 -65
- data/lib/rest-core/middleware/timeout/{eventmachine_timer.rb → timer_em.rb} +3 -1
- data/lib/rest-core/middleware/timeout/timer_thread.rb +36 -0
- data/lib/rest-core/patch/multi_json.rb +8 -0
- data/lib/rest-core/test.rb +3 -12
- data/lib/rest-core/util/json.rb +65 -0
- data/lib/rest-core/util/parse_query.rb +2 -2
- data/lib/rest-core/version.rb +1 -1
- data/lib/rest-core/wrapper.rb +16 -16
- data/rest-core.gemspec +28 -27
- data/test/test_auth_basic.rb +14 -10
- data/test/test_builder.rb +7 -7
- data/test/test_cache.rb +126 -37
- data/test/test_client.rb +3 -1
- data/test/test_client_oauth1.rb +2 -3
- data/test/test_default_query.rb +17 -23
- data/test/test_em_http_request.rb +146 -0
- data/test/test_error_detector.rb +0 -1
- data/test/test_error_handler.rb +44 -0
- data/test/test_follow_redirect.rb +17 -19
- data/test/test_json_request.rb +28 -0
- data/test/test_json_response.rb +51 -0
- data/test/test_oauth1_header.rb +4 -4
- data/test/test_payload.rb +20 -12
- data/test/test_simple.rb +14 -0
- data/test/test_timeout.rb +11 -19
- data/test/test_universal.rb +5 -5
- data/test/test_wrapper.rb +19 -13
- metadata +28 -29
- data/doc/ToC.md +0 -7
- data/doc/dependency.md +0 -4
- data/doc/design.md +0 -4
- data/example/auto.rb +0 -51
- data/example/coolio.rb +0 -21
- data/example/eventmachine.rb +0 -30
- data/example/rest-client.rb +0 -16
- data/lib/rest-core/app/abstract/async_fiber.rb +0 -13
- data/lib/rest-core/app/auto.rb +0 -23
- data/lib/rest-core/app/coolio-async.rb +0 -32
- data/lib/rest-core/app/coolio-fiber.rb +0 -30
- data/lib/rest-core/app/coolio.rb +0 -9
- data/lib/rest-core/app/em-http-request-async.rb +0 -37
- data/lib/rest-core/app/em-http-request-fiber.rb +0 -45
- data/lib/rest-core/app/em-http-request.rb +0 -9
- data/lib/rest-core/app/rest-client.rb +0 -41
- data/lib/rest-core/middleware/json_decode.rb +0 -93
- data/lib/rest-core/middleware/timeout/coolio_timer.rb +0 -10
- data/pending/test_multi.rb +0 -123
- data/pending/test_test_util.rb +0 -86
- data/test/test_json_decode.rb +0 -24
data/.travis.yml
CHANGED
@@ -1,11 +1,10 @@
|
|
1
1
|
before_install: 'git submodule update --init'
|
2
|
-
script: '
|
2
|
+
script: 'ruby -r bundler/setup -S rake test'
|
3
|
+
|
4
|
+
env:
|
5
|
+
- 'RBXOPT=-X19'
|
3
6
|
|
4
7
|
rvm:
|
5
|
-
- 1.8.7
|
6
|
-
- 1.9.2
|
7
8
|
- 1.9.3
|
8
|
-
- rbx-
|
9
|
-
-
|
10
|
-
- jruby-18mode
|
11
|
-
- jruby-19mode
|
9
|
+
- rbx-head
|
10
|
+
- jruby-head
|
data/CHANGES.md
CHANGED
@@ -1,5 +1,142 @@
|
|
1
1
|
# CHANGES
|
2
2
|
|
3
|
+
## rest-core 2.0.0
|
4
|
+
|
5
|
+
This is a major release which introduces some incompatible changes.
|
6
|
+
This is intended to cleanup some internal implementation and introduce
|
7
|
+
a new mechanism to handle multiple requests concurrently, avoiding needless
|
8
|
+
block.
|
9
|
+
|
10
|
+
Before we go into detail, here's who can upgrade without changing anything,
|
11
|
+
and who should make a few adjustments in their code:
|
12
|
+
|
13
|
+
* If you're only using rest-more, e.g. `RC::Facebook` or `RC::Twitter`, etc.,
|
14
|
+
you don't have to change anything. This won't affect rest-more users.
|
15
|
+
(except that JsonDecode is renamed to JsonResponse, and json_decode is
|
16
|
+
renamed to json_response.)
|
17
|
+
|
18
|
+
* If you're only using rest-core's built in middlewares to build your own
|
19
|
+
clients, you don't have to change anything as well. All the hard works are
|
20
|
+
done in rest-core. (except that ErrorHandler works a bit differently now.
|
21
|
+
We'll talk about detail later.)
|
22
|
+
|
23
|
+
* If you're building your own middlewares, then you are the ones who need to
|
24
|
+
make changes. `RC::ASYNC` is changed to a flag to mean whether the callback
|
25
|
+
should be called directly, or only after resuming from the future (fiber
|
26
|
+
or thread). And now you have always to get the response from `yield`, that
|
27
|
+
is, you're forced to pass a callback to `call`.
|
28
|
+
|
29
|
+
This might be a bit user unfriendly at first glimpse, but it would much
|
30
|
+
simplify the internal structure of rest-core, because in the middlewares,
|
31
|
+
you don't have to worry if the user would pass a callback or not, branching
|
32
|
+
everywhere to make it work both synchronously and asynchronously.
|
33
|
+
|
34
|
+
Also, the old fiber based asynchronous HTTP client is removed, in favor
|
35
|
+
of the new _future_ based approach. The new one is more like a superset
|
36
|
+
of the old one, which have anything the old one can provide. Yet internally
|
37
|
+
it works a lot differently. They are both synchronous to the outsides,
|
38
|
+
but while the old one is also synchronous inside, the new one is
|
39
|
+
asynchronous inside, just like the purely asynchronous HTTP client.
|
40
|
+
|
41
|
+
That is, internally, it's always asynchronously, and fiber/async didn't
|
42
|
+
make much difference here now. This is also the reason why I removed
|
43
|
+
the old fiber one. This would make the middleware implementation much
|
44
|
+
easier, considering much fewer possible cases.
|
45
|
+
|
46
|
+
If you don't really understand what above does mean, then just remember,
|
47
|
+
now we ask all middlewares work asynchronously. You have always to work
|
48
|
+
with callbacks which passed along in `app.call(env){ |response| }`
|
49
|
+
That's it.
|
50
|
+
|
51
|
+
So what's the most important improvement? From now on, we have only two
|
52
|
+
modes. One is callback mode, in which case `env[ASYNC]` would be set, and
|
53
|
+
the callback would be called. No exception would be raised in this case.
|
54
|
+
If there's an exception, then it would be passed to the block instead.
|
55
|
+
|
56
|
+
The other mode, which is synchronous, is achieved by the futures. We have
|
57
|
+
two different kinds of futures for now, one is thread based, and the other
|
58
|
+
is fiber based. For RestClient, thread based future would be used. For
|
59
|
+
EventMachine, depending on the context, if the future is created on the
|
60
|
+
main thread, then it would assume it's also wrapped inside a fiber. Since,
|
61
|
+
we can never block the event loop! If you're not calling it in a thread,
|
62
|
+
you must call it in a fiber. But if you're calling it in a thread, then
|
63
|
+
the thread based future would be picked. This is because otherwise it won't
|
64
|
+
work well exchanging information around threads and fibers.
|
65
|
+
|
66
|
+
In short, rest-core would run concurrently in all contexts, archived by
|
67
|
+
either threads or fibers depending on the context, and it would pick the
|
68
|
+
right strategy for you.
|
69
|
+
|
70
|
+
You can see [use-cases.rb][] for all possible use cases.
|
71
|
+
|
72
|
+
It's a bit outdated, but you can also checkout my blog post.
|
73
|
+
[rest-core 2.0 roadmap, thunk based response][post]
|
74
|
+
(p.s. now thunk is renamed to future)
|
75
|
+
|
76
|
+
[use-cases.rb]: https://github.com/cardinalblue/rest-core/blob/master/example/use-cases.rb
|
77
|
+
[post]: http://blogger.godfat.org/2012/06/rest-core-20-roadmap-thunk-based.html
|
78
|
+
|
79
|
+
### Incompatible changes
|
80
|
+
|
81
|
+
* [JsonDecode] is renamed to JsonResponse, and json_decode is also renamed
|
82
|
+
to json_response.
|
83
|
+
* [Json] Now you can use `Json.decode` and `Json.encode` to parse and
|
84
|
+
generate JSONs, instead of `JsonDecode.json_decode`.
|
85
|
+
* [Cache] Support for "cache.post" is removed.
|
86
|
+
* [Cache] The cache key is changed accordingly to support cache for headers
|
87
|
+
and HTTP status. If you don't have persistent cache, this doesn't matter.
|
88
|
+
|
89
|
+
* [EmHttpRequestFiber] is removed in favor of `EmHttpRequest`
|
90
|
+
* cool.io support is removed.
|
91
|
+
* You must provide a block to `app.call(env){ ... }`.
|
92
|
+
* Rename Wrapper#default_app to Wrapper#default_engine
|
93
|
+
|
94
|
+
### Enhancement
|
95
|
+
|
96
|
+
* The default engine is changed from `RestClient` to `Auto`, which would
|
97
|
+
be using `EmHttpRequest` under the context of a event loop, while
|
98
|
+
use `RestClient` in other context as before.
|
99
|
+
|
100
|
+
* `RestCore.eagerload` is introduced to load all constants eagerly. You can
|
101
|
+
use this before loading the application to avoid thread-safety issue in
|
102
|
+
autoload. For the lazies.
|
103
|
+
|
104
|
+
* [JsonResponse] This is originally JsonDecode, and now we prefer multi_json
|
105
|
+
first, yajl-ruby second, lastly json.
|
106
|
+
* [JsonResponse] give JsonResponse a default header Accept: application/json,
|
107
|
+
thanks @ayamomiji
|
108
|
+
* [JsonRequest] This middleware would encode your payload into a JSON.
|
109
|
+
* [CommonLogger] Now we log the request method as well.
|
110
|
+
* [DefaultPayload] Accept arbitrary payload.
|
111
|
+
* [DefaultQuery] Now before merging queries, converting every single key into
|
112
|
+
a string. This allows you to use :symbols for default query.
|
113
|
+
|
114
|
+
* [ErrorHandler] So now ErrorHandler is working differently. It would first
|
115
|
+
try to see if `env[FAIL]` has any exception in it. If there is, then raise
|
116
|
+
it. Otherwise it would call error_handler and expect it to generate an
|
117
|
+
error object. If the error object is an exception, then raise it. If it's
|
118
|
+
not, then it would merge it into `env[FAIL]`. On the other hand, in the
|
119
|
+
case of using callbacks instead of futures, it would pass the exception
|
120
|
+
as the `env[RESPONSE_BODY]` instead. The reason is that you can't raise
|
121
|
+
an exception asynchronously and handle it safely.
|
122
|
+
|
123
|
+
* [Cache] Now response headers and HTTP status are also cached.
|
124
|
+
* [Cache] Not only GET requests are cached, HEAD and OPTIONS are cached too.
|
125
|
+
* [Cache] The cache key is also respecting the request headers too. Suppose
|
126
|
+
you're making a request with different Accept header.
|
127
|
+
|
128
|
+
* [Client] Add Client#wait which would block until all requests for this
|
129
|
+
particular client are done.
|
130
|
+
|
131
|
+
### Bugs fixes
|
132
|
+
|
133
|
+
* [Middleware] Sort the query before generating the request URI, making
|
134
|
+
sure the order is always the same.
|
135
|
+
* [Middleware] The middleware could have no members at all.
|
136
|
+
* [ParseQuery] The fallback function for the absence of Rack is fixed.
|
137
|
+
* [Auto] Only use EmHttpRequest if em-http-request is loaded,
|
138
|
+
thanks @ayamomiji
|
139
|
+
|
3
140
|
## rest-core 1.0.3 -- 2012-08-15
|
4
141
|
|
5
142
|
### Enhancement
|
data/Gemfile
CHANGED
data/README.md
CHANGED
@@ -6,8 +6,6 @@ Lin Jen-Shin ([godfat][]) had given a talk about rest-core on
|
|
6
6
|
[RubyConf Taiwan 2011][talk]. The slide is in English, but the
|
7
7
|
talk is in Mandarin.
|
8
8
|
|
9
|
-
You can also read some other topics at [doc](https://github.com/cardinalblue/rest-core/blob/master/doc/ToC.md).
|
10
|
-
|
11
9
|
[godfat]: https://github.com/godfat
|
12
10
|
[talk]: http://rubyconf.tw/2011/#6
|
13
11
|
|
@@ -24,47 +22,45 @@ Modular Ruby clients interface for REST APIs
|
|
24
22
|
|
25
23
|
There has been an explosion in the number of REST APIs available today.
|
26
24
|
To address the need for a way to access these APIs easily and elegantly,
|
27
|
-
we have developed
|
25
|
+
we have developed rest-core, which consists of composable middleware
|
28
26
|
that allows you to build a REST client for any REST API. Or in the case of
|
29
27
|
common APIs such as Facebook, Github, and Twitter, you can simply use the
|
30
28
|
dedicated clients provided by [rest-more][].
|
31
29
|
|
32
|
-
[rest-core]: https://github.com/cardinalblue/rest-core
|
33
30
|
[rest-more]: https://github.com/cardinalblue/rest-more
|
34
31
|
|
35
32
|
## FEATURES:
|
36
33
|
|
37
34
|
* Modular interface for REST clients similar to WSGI/Rack for servers.
|
38
|
-
*
|
35
|
+
* Concurrent requests with synchronous or asynchronous interfaces with
|
36
|
+
fibers or threads are both supported.
|
39
37
|
|
40
38
|
## REQUIREMENTS:
|
41
39
|
|
42
40
|
### Mandatory:
|
43
41
|
|
44
|
-
* MRI (official CRuby) 1.
|
42
|
+
* MRI (official CRuby) 1.9.3, Rubinius 1.9 and JRuby 1.9
|
45
43
|
* gem rest-client
|
46
44
|
|
47
45
|
### Optional:
|
48
46
|
|
49
|
-
* Fibers only work on Ruby 1.9+
|
50
47
|
* gem [em-http-request][] (if using eventmachine)
|
51
|
-
* gem
|
52
|
-
|
48
|
+
* gem json or yajl-ruby, or multi_json (if `JsonResponse` or
|
49
|
+
`JsonRequest` middlewares are used)
|
53
50
|
|
54
51
|
[em-http-request]: https://github.com/igrigorik/em-http-request
|
55
|
-
[cool.io-http]: https://github.com/godfat/cool.io-http
|
56
52
|
|
57
53
|
## INSTALLATION:
|
58
54
|
|
59
55
|
``` shell
|
60
|
-
|
56
|
+
gem install rest-core
|
61
57
|
```
|
62
58
|
|
63
59
|
Or if you want development version, put this in Gemfile:
|
64
60
|
|
65
61
|
``` ruby
|
66
|
-
|
67
|
-
|
62
|
+
gem 'rest-core', :git => 'git://github.com/cardinalblue/rest-core.git',
|
63
|
+
:submodules => true
|
68
64
|
```
|
69
65
|
|
70
66
|
If you just want to use Facebook or Twitter clients, please take a look at
|
@@ -74,235 +70,231 @@ If you just want to use Facebook or Twitter clients, please take a look at
|
|
74
70
|
|
75
71
|
## Build Your Own Clients:
|
76
72
|
|
77
|
-
You can use `RestCore::Builder` to build your own dedicated
|
73
|
+
You can use `RestCore::Builder` to build your own dedicated clients.
|
74
|
+
Note that `RC` is an alias of `RestCore`
|
78
75
|
|
79
76
|
``` ruby
|
80
|
-
|
81
|
-
|
82
|
-
|
83
|
-
|
84
|
-
|
85
|
-
|
86
|
-
|
87
|
-
use s::Cache , nil, 3600
|
88
|
-
run s::RestClient # the simplest and easier HTTP client
|
89
|
-
end
|
77
|
+
require 'rest-core'
|
78
|
+
YourClient = RC::Builder.client do
|
79
|
+
use RC::DefaultSite , 'https://api.github.com/users/'
|
80
|
+
use RC::JsonResponse, true
|
81
|
+
use RC::CommonLogger, method(:puts)
|
82
|
+
use RC::Cache , nil, 3600
|
83
|
+
end
|
90
84
|
```
|
91
85
|
|
92
86
|
And use it with per-instance basis (clients could have different
|
93
87
|
configuration, e.g. different cache time or timeout time):
|
94
88
|
|
95
89
|
``` ruby
|
96
|
-
|
97
|
-
|
98
|
-
|
90
|
+
client = YourClient.new(:cache => {})
|
91
|
+
client.get('cardinalblue') # cache miss
|
92
|
+
client.get('cardinalblue') # cache hit
|
99
93
|
|
100
|
-
|
101
|
-
|
102
|
-
|
94
|
+
client.site = 'http://github.com/api/v2/json/user/show/'
|
95
|
+
client.get('cardinalblue') # cache miss
|
96
|
+
client.get('cardinalblue') # cache hit
|
103
97
|
```
|
104
98
|
|
105
|
-
|
106
|
-
|
107
|
-
for concepts.
|
108
|
-
|
109
|
-
[example/rest-client.rb]: https://github.com/cardinalblue/rest-core/blob/master/example/rest-client.rb
|
110
|
-
[rest-more]: https://github.com/cardinalblue/rest-more
|
111
|
-
[slides]: http://www.godfat.org/slide/2011-08-27-rest-core.html
|
112
|
-
[rubyconf.tw]: http://rubyconf.tw/2011/#6
|
113
|
-
|
114
|
-
## Asynchronous HTTP Requests:
|
115
|
-
|
116
|
-
I/O bound operations shouldn't be blocking the CPU! If you have a reactor,
|
117
|
-
i.e. event loop, you should take the advantage of that to make HTTP requests
|
118
|
-
not block the whole process/thread. For now, we support eventmachine and
|
119
|
-
cool.io. Below is an example for eventmachine:
|
99
|
+
You can also make concurrent requests easily:
|
100
|
+
(see "Advanced Concurrent HTTP Requests -- Embrace the Future" for detail)
|
120
101
|
|
121
102
|
``` ruby
|
122
|
-
|
123
|
-
|
124
|
-
|
125
|
-
|
126
|
-
use s::DefaultSite , 'https://api.github.com/users/'
|
127
|
-
use s::JsonDecode , true
|
128
|
-
use s::CommonLogger, method(:puts)
|
129
|
-
use s::Cache , nil, 3600
|
130
|
-
run s::EmHttpRequest
|
131
|
-
end
|
103
|
+
a = [client.get('cardinalblue')['name'], client.get('godfat')['name']]
|
104
|
+
puts "It's not blocking... but doing concurrent requests underneath"
|
105
|
+
p a # here we want the values, so it blocks here
|
106
|
+
puts "DONE"
|
132
107
|
```
|
133
108
|
|
134
|
-
|
135
|
-
available. That is the block is the callback for the request.
|
109
|
+
Callback mode also available:
|
136
110
|
|
137
111
|
``` ruby
|
138
|
-
|
139
|
-
|
140
|
-
|
141
|
-
|
142
|
-
EM.stop
|
143
|
-
}
|
144
|
-
puts "It's not blocking..."
|
145
|
-
}
|
112
|
+
client.get('cardinalblue'){ |v| p v }
|
113
|
+
puts "It's not blocking... but doing concurrent requests underneath"
|
114
|
+
client.wait # we block here to wait for the request done
|
115
|
+
puts "DONE"
|
146
116
|
```
|
147
117
|
|
148
|
-
|
149
|
-
|
150
|
-
|
151
|
-
If you don't understand what does this mean, you can take a look at
|
152
|
-
[em-synchrony][]. It's basically the same idea.
|
118
|
+
Runnable example is at: [example/simple.rb][]. Please see [rest-more][]
|
119
|
+
for more complex examples to build clients, and [slides][] from
|
120
|
+
[rubyconf.tw/2011][rubyconf.tw] for concepts.
|
153
121
|
|
154
|
-
|
155
|
-
|
156
|
-
|
157
|
-
|
158
|
-
EM.stop
|
159
|
-
}.resume
|
160
|
-
puts "It's not blocking..."
|
161
|
-
}
|
162
|
-
```
|
163
|
-
|
164
|
-
[em-synchrony]: https://github.com/igrigorik/em-synchrony
|
165
|
-
|
166
|
-
Runnable example is here: [example/eventmachine.rb][].
|
167
|
-
You can also make multi-requests synchronously like this:
|
168
|
-
|
169
|
-
``` ruby
|
170
|
-
EM.run{
|
171
|
-
Fiber.new{
|
172
|
-
fiber = Fiber.current
|
173
|
-
result = {}
|
174
|
-
client.get('cardinalblue'){ |response|
|
175
|
-
result[0] = response
|
176
|
-
fiber.resume(result) if result.size == 2
|
177
|
-
}
|
178
|
-
client.get('cardinalblue'){ |response|
|
179
|
-
result[1] = response
|
180
|
-
fiber.resume(result) if result.size == 2
|
181
|
-
}
|
182
|
-
p Fiber.yield
|
183
|
-
EM.stop
|
184
|
-
}.resume
|
185
|
-
puts "It's not blocking..."
|
186
|
-
}
|
187
|
-
```
|
188
|
-
|
189
|
-
Runnable example is here: [example/multi.rb][].
|
190
|
-
|
191
|
-
[example/eventmachine.rb]: https://github.com/cardinalblue/rest-core/blob/master/example/eventmachine.rb
|
192
|
-
[example/multi.rb]: https://github.com/cardinalblue/rest-core/blob/master/example/multi.rb
|
193
|
-
|
194
|
-
## Supported HTTP clients:
|
122
|
+
[example/simple.rb]: https://github.com/cardinalblue/rest-core/blob/master/example/simple.rb
|
123
|
+
[rest-more]: https://github.com/cardinalblue/rest-more
|
124
|
+
[slides]: http://www.godfat.org/slide/2011-08-27-rest-core.html
|
125
|
+
[rubyconf.tw]: http://rubyconf.tw/2011/#6
|
195
126
|
|
196
|
-
|
197
|
-
|
198
|
-
* `
|
199
|
-
* `
|
200
|
-
|
127
|
+
## List of built-in Middlewares:
|
128
|
+
|
129
|
+
* `RC::AuthBasic`
|
130
|
+
* `RC::Bypass`
|
131
|
+
* `RC::Cache`
|
132
|
+
* `RC::CommonLogger`
|
133
|
+
* `RC::DefaultHeaders`
|
134
|
+
* `RC::DefaultPayload`
|
135
|
+
* `RC::DefaultQuery`
|
136
|
+
* `RC::DefaultSite`
|
137
|
+
* `RC::Defaults`
|
138
|
+
* `RC::ErrorDetector`
|
139
|
+
* `RC::ErrorDetectorHttp`
|
140
|
+
* `RC::ErrorHandler`
|
141
|
+
* `RC::FollowRedirect`
|
142
|
+
* `RC::JsonRequest`
|
143
|
+
* `RC::JsonResponse`
|
144
|
+
* `RC::Oauth1Header`
|
145
|
+
* `RC::Oauth2Header`
|
146
|
+
* `RC::Oauth2Query`
|
147
|
+
* `RC::Timeout`
|
201
148
|
|
202
149
|
## Build Your Own Middlewares:
|
203
150
|
|
204
151
|
To be added.
|
205
152
|
|
206
|
-
##
|
207
|
-
|
208
|
-
To be added.
|
209
|
-
|
210
|
-
## rest-core users:
|
211
|
-
|
212
|
-
* [topcoder](https://github.com/miaout17/topcoder)
|
213
|
-
* [s2sync](https://github.com/brucehsu/s2sync)
|
214
|
-
* [s2sync_web](https://github.com/brucehsu/s2sync_web)
|
153
|
+
## Advanced Concurrent HTTP Requests -- Embrace the Future
|
215
154
|
|
216
|
-
|
155
|
+
### The Interface
|
217
156
|
|
218
|
-
|
157
|
+
There are a number of different ways to make concurrent requests in
|
158
|
+
rest-core. They could be roughly categorized to two different forms.
|
159
|
+
One is using the well known callbacks, while the other one is using
|
160
|
+
through a technique called [future][]. Basically, it means it would
|
161
|
+
return you a promise, which would eventually become the real value
|
162
|
+
(response here) you were asking for whenever you really want it.
|
163
|
+
Otherwise, the program keeps running until the value is evaluated,
|
164
|
+
and blocks there if the computation (response) hasn't been done yet.
|
165
|
+
If the computation is already done, then it would simply return you
|
166
|
+
the result.
|
219
167
|
|
220
|
-
|
168
|
+
Here's a very simple example for using futures:
|
221
169
|
|
222
|
-
|
170
|
+
``` ruby
|
171
|
+
require 'rest-core'
|
172
|
+
YourClient = RC::Builder.client do
|
173
|
+
use RC::DefaultSite , 'https://api.github.com/users/'
|
174
|
+
use RC::JsonResponse, true
|
175
|
+
use RC::CommonLogger, method(:puts)
|
176
|
+
end
|
177
|
+
|
178
|
+
client = YourClient.new
|
179
|
+
puts "rest-client with threads doing concurrent requests"
|
180
|
+
a = [client.get('cardinalblue')['name'], client.get('godfat')['name']]
|
181
|
+
puts "It's not blocking... but doing concurrent requests underneath"
|
182
|
+
p a # here we want the values, so it blocks here
|
183
|
+
puts "DONE"
|
184
|
+
```
|
223
185
|
|
224
|
-
|
186
|
+
And here's a corresponded version for using callbacks:
|
225
187
|
|
226
|
-
|
227
|
-
|
188
|
+
``` ruby
|
189
|
+
require 'rest-core'
|
190
|
+
YourClient = RC::Builder.client do
|
191
|
+
use RC::DefaultSite , 'https://api.github.com/users/'
|
192
|
+
use RC::JsonResponse, true
|
193
|
+
use RC::CommonLogger, method(:puts)
|
194
|
+
end
|
195
|
+
|
196
|
+
client = YourClient.new
|
197
|
+
puts "rest-client with threads doing concurrent requests"
|
198
|
+
client.get('cardinalblue'){ |v|
|
199
|
+
p v['name']
|
200
|
+
}.
|
201
|
+
get('godfat'){ |v|
|
202
|
+
p v['name']
|
203
|
+
}
|
204
|
+
puts "It's not blocking... but doing concurrent requests underneath"
|
205
|
+
client.wait # until all requests are done
|
206
|
+
puts "DONE"
|
207
|
+
```
|
228
208
|
|
229
|
-
|
230
|
-
For instance, `RestCore::RestClient` is an HTTP client which uses
|
231
|
-
rest-client gem (`::RestClient`) to make HTTP requests.
|
209
|
+
You can pick whatever works for you.
|
232
210
|
|
233
|
-
|
234
|
-
For instance, `RestCore::DefaultSite` is a middleware which would add
|
235
|
-
default site URL in front of the request URI if it is not started with
|
236
|
-
http://, thus you can do this: `RestCore::Facebook.get('4')` without
|
237
|
-
specifying where the site (Facebook) it is.
|
211
|
+
[future]: http://en.wikipedia.org/wiki/Futures_and_promises
|
238
212
|
|
239
|
-
|
240
|
-
middlewares into another middleware. Currently, it's used in
|
241
|
-
`RestCore::Buidler` and `RestCore::Cache`.
|
213
|
+
### What Concurrency Model to Choose?
|
242
214
|
|
243
|
-
|
244
|
-
|
215
|
+
In the above example, we're using rest-client with threads, which works
|
216
|
+
for most of cases. But you might also want to use em-http-request with
|
217
|
+
EventMachine, which is using a faster HTTP parser. In theory, it should
|
218
|
+
be much more efficient than rest-client and threads.
|
245
219
|
|
246
|
-
|
247
|
-
|
220
|
+
To pick em-http-request, you must run the requests inside the EventMachine's
|
221
|
+
event loop, and also wrap your request with either a thread or a fiber,
|
222
|
+
because we can't block the event loop and ask em-http-request to finish
|
223
|
+
its job making requests.
|
248
224
|
|
249
|
-
|
250
|
-
_client_ by `RestCore::Builder`. It contains a number of convenient
|
251
|
-
functions which is generally useful.
|
225
|
+
Here's an example of using em-http-request with threads:
|
252
226
|
|
253
|
-
|
254
|
-
|
255
|
-
|
227
|
+
``` ruby
|
228
|
+
require 'em-http-request'
|
229
|
+
require 'rest-core'
|
230
|
+
YourClient = RC::Builder.client do
|
231
|
+
use RC::DefaultSite , 'https://api.github.com/users/'
|
232
|
+
use RC::JsonResponse, true
|
233
|
+
use RC::CommonLogger, method(:puts)
|
234
|
+
end
|
235
|
+
|
236
|
+
client = YourClient.new
|
237
|
+
puts "eventmachine with threads doing concurrent requests"
|
238
|
+
EM.run{
|
239
|
+
Thread.new{
|
240
|
+
p [client.get('cardinalblue')['name'], client.get('godfat')['name']]
|
241
|
+
puts "DONE"
|
242
|
+
EM.stop
|
243
|
+
}
|
244
|
+
puts "It's not blocking... but doing concurrent requests underneath"
|
245
|
+
}
|
246
|
+
```
|
256
247
|
|
257
|
-
|
258
|
-
response. It's mostly seen in `@app.call(env)` See other explanation
|
259
|
-
such as `env[RestCore::REQUEST_METHOD]` for more detail.
|
248
|
+
And here's an example of using em-http-request with fibers:
|
260
249
|
|
261
|
-
|
262
|
-
|
263
|
-
|
250
|
+
``` ruby
|
251
|
+
require 'fiber' # remember to require fiber first,
|
252
|
+
require 'em-http-request' # or rest-core won't pick fibers
|
253
|
+
require 'rest-core'
|
254
|
+
YourClient = RC::Builder.client do
|
255
|
+
use RC::DefaultSite , 'https://api.github.com/users/'
|
256
|
+
use RC::JsonResponse, true
|
257
|
+
use RC::CommonLogger, method(:puts)
|
258
|
+
end
|
259
|
+
|
260
|
+
client = YourClient.new
|
261
|
+
puts "eventmachine with fibers doing concurrent requests"
|
262
|
+
EM.run{
|
263
|
+
Fiber.new{
|
264
|
+
p [client.get('cardinalblue')['name'], client.get('godfat')['name']]
|
265
|
+
puts "DONE"
|
266
|
+
EM.stop
|
267
|
+
}
|
268
|
+
puts "It's not blocking... but doing concurrent requests underneath"
|
269
|
+
}
|
270
|
+
```
|
264
271
|
|
265
|
-
|
266
|
-
|
267
|
-
|
268
|
-
simply `"4"`. In the case of built-in Facebook client, the
|
269
|
-
`RestCore::DefaultSite` middleware would take care of the site.
|
272
|
+
As you can see, both of them are quite similar to each other, because the
|
273
|
+
idea behind the scene is the same. If you don't know what concurrency model
|
274
|
+
to pick, start with rest-client since it's the easiest one to setup.
|
270
275
|
|
271
|
-
|
272
|
-
|
273
|
-
|
274
|
-
|
276
|
+
A full runnable example is at: [example/multi.rb][]. If you want to know
|
277
|
+
all the possible use cases, you can also see: [example/use-cases.rb][]. It's
|
278
|
+
also served as a test for each possible combinations, so it's quite complex
|
279
|
+
and complete.
|
275
280
|
|
276
|
-
|
277
|
-
values are payload values. Both keys and values' type should be String,
|
278
|
-
not Symbol. Values with nil or false would be ignored. Both keys and values
|
279
|
-
would be escaped automatically.
|
281
|
+
[example/multi.rb]: https://github.com/cardinalblue/rest-core/blob/master/example/multi.rb
|
280
282
|
|
281
|
-
|
282
|
-
values are header values. Both keys and values' type should be String,
|
283
|
-
not Symbol. Values with nil or false would be ignored.
|
283
|
+
[example/use-cases.rb]: https://github.com/cardinalblue/rest-core/blob/master/example/use-cases.rb
|
284
284
|
|
285
|
-
|
286
|
-
Might be nil if there's no response or not yet making HTTP request.
|
285
|
+
## rest-core users:
|
287
286
|
|
288
|
-
*
|
289
|
-
|
290
|
-
|
287
|
+
* [topcoder](https://github.com/miaout17/topcoder)
|
288
|
+
* [s2sync](https://github.com/brucehsu/s2sync)
|
289
|
+
* [s2sync_web](https://github.com/brucehsu/s2sync_web)
|
291
290
|
|
292
|
-
|
293
|
-
for the response headers. Both keys and values' type should be String.
|
291
|
+
## Powered sites:
|
294
292
|
|
295
|
-
*
|
296
|
-
indicates that if we're only asking for modified `env`, instead of making
|
297
|
-
real requests. It's used to ask for the real request URI, etc.
|
293
|
+
* [PicCollage](http://pic-collage.com/)
|
298
294
|
|
299
|
-
|
300
|
-
could be any objects, it's handled by `RestCore::ErrorDetector` or any
|
301
|
-
other custom _middleware_.
|
295
|
+
## CHANGES:
|
302
296
|
|
303
|
-
*
|
304
|
-
could be any objects, it's handled by `RestCore::CommonLogger` or
|
305
|
-
any other custom _middleware_.
|
297
|
+
* [CHANGES](https://github.com/cardinalblue/rest-core/blob/master/CHANGES.md)
|
306
298
|
|
307
299
|
## CONTRIBUTORS:
|
308
300
|
|