rest-core 1.0.3 → 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/.travis.yml +6 -7
- data/CHANGES.md +137 -0
- data/Gemfile +1 -1
- data/README.md +183 -191
- data/TODO.md +5 -8
- data/example/multi.rb +31 -24
- data/example/simple.rb +28 -0
- data/example/use-cases.rb +194 -0
- data/lib/rest-core.rb +26 -19
- data/lib/rest-core/builder.rb +2 -2
- data/lib/rest-core/client.rb +40 -27
- data/lib/rest-core/client/universal.rb +16 -13
- data/lib/rest-core/client_oauth1.rb +5 -5
- data/lib/rest-core/engine/auto.rb +25 -0
- data/lib/rest-core/{app → engine}/dry.rb +1 -2
- data/lib/rest-core/engine/em-http-request.rb +39 -0
- data/lib/rest-core/engine/future/future.rb +106 -0
- data/lib/rest-core/engine/future/future_fiber.rb +39 -0
- data/lib/rest-core/engine/future/future_thread.rb +29 -0
- data/lib/rest-core/engine/rest-client.rb +56 -0
- data/lib/rest-core/middleware.rb +27 -5
- data/lib/rest-core/middleware/auth_basic.rb +5 -5
- data/lib/rest-core/middleware/bypass.rb +2 -2
- data/lib/rest-core/middleware/cache.rb +67 -54
- data/lib/rest-core/middleware/common_logger.rb +5 -8
- data/lib/rest-core/middleware/default_headers.rb +2 -2
- data/lib/rest-core/middleware/default_payload.rb +26 -2
- data/lib/rest-core/middleware/default_query.rb +4 -2
- data/lib/rest-core/middleware/default_site.rb +8 -6
- data/lib/rest-core/middleware/error_detector.rb +9 -16
- data/lib/rest-core/middleware/error_handler.rb +25 -11
- data/lib/rest-core/middleware/follow_redirect.rb +11 -14
- data/lib/rest-core/middleware/json_request.rb +19 -0
- data/lib/rest-core/middleware/json_response.rb +28 -0
- data/lib/rest-core/middleware/oauth1_header.rb +2 -7
- data/lib/rest-core/middleware/oauth2_header.rb +4 -7
- data/lib/rest-core/middleware/oauth2_query.rb +2 -2
- data/lib/rest-core/middleware/timeout.rb +21 -65
- data/lib/rest-core/middleware/timeout/{eventmachine_timer.rb → timer_em.rb} +3 -1
- data/lib/rest-core/middleware/timeout/timer_thread.rb +36 -0
- data/lib/rest-core/patch/multi_json.rb +8 -0
- data/lib/rest-core/test.rb +3 -12
- data/lib/rest-core/util/json.rb +65 -0
- data/lib/rest-core/util/parse_query.rb +2 -2
- data/lib/rest-core/version.rb +1 -1
- data/lib/rest-core/wrapper.rb +16 -16
- data/rest-core.gemspec +28 -27
- data/test/test_auth_basic.rb +14 -10
- data/test/test_builder.rb +7 -7
- data/test/test_cache.rb +126 -37
- data/test/test_client.rb +3 -1
- data/test/test_client_oauth1.rb +2 -3
- data/test/test_default_query.rb +17 -23
- data/test/test_em_http_request.rb +146 -0
- data/test/test_error_detector.rb +0 -1
- data/test/test_error_handler.rb +44 -0
- data/test/test_follow_redirect.rb +17 -19
- data/test/test_json_request.rb +28 -0
- data/test/test_json_response.rb +51 -0
- data/test/test_oauth1_header.rb +4 -4
- data/test/test_payload.rb +20 -12
- data/test/test_simple.rb +14 -0
- data/test/test_timeout.rb +11 -19
- data/test/test_universal.rb +5 -5
- data/test/test_wrapper.rb +19 -13
- metadata +28 -29
- data/doc/ToC.md +0 -7
- data/doc/dependency.md +0 -4
- data/doc/design.md +0 -4
- data/example/auto.rb +0 -51
- data/example/coolio.rb +0 -21
- data/example/eventmachine.rb +0 -30
- data/example/rest-client.rb +0 -16
- data/lib/rest-core/app/abstract/async_fiber.rb +0 -13
- data/lib/rest-core/app/auto.rb +0 -23
- data/lib/rest-core/app/coolio-async.rb +0 -32
- data/lib/rest-core/app/coolio-fiber.rb +0 -30
- data/lib/rest-core/app/coolio.rb +0 -9
- data/lib/rest-core/app/em-http-request-async.rb +0 -37
- data/lib/rest-core/app/em-http-request-fiber.rb +0 -45
- data/lib/rest-core/app/em-http-request.rb +0 -9
- data/lib/rest-core/app/rest-client.rb +0 -41
- data/lib/rest-core/middleware/json_decode.rb +0 -93
- data/lib/rest-core/middleware/timeout/coolio_timer.rb +0 -10
- data/pending/test_multi.rb +0 -123
- data/pending/test_test_util.rb +0 -86
- data/test/test_json_decode.rb +0 -24
data/.travis.yml
CHANGED
@@ -1,11 +1,10 @@
|
|
1
1
|
before_install: 'git submodule update --init'
|
2
|
-
script: '
|
2
|
+
script: 'ruby -r bundler/setup -S rake test'
|
3
|
+
|
4
|
+
env:
|
5
|
+
- 'RBXOPT=-X19'
|
3
6
|
|
4
7
|
rvm:
|
5
|
-
- 1.8.7
|
6
|
-
- 1.9.2
|
7
8
|
- 1.9.3
|
8
|
-
- rbx-
|
9
|
-
-
|
10
|
-
- jruby-18mode
|
11
|
-
- jruby-19mode
|
9
|
+
- rbx-head
|
10
|
+
- jruby-head
|
data/CHANGES.md
CHANGED
@@ -1,5 +1,142 @@
|
|
1
1
|
# CHANGES
|
2
2
|
|
3
|
+
## rest-core 2.0.0
|
4
|
+
|
5
|
+
This is a major release which introduces some incompatible changes.
|
6
|
+
This is intended to cleanup some internal implementation and introduce
|
7
|
+
a new mechanism to handle multiple requests concurrently, avoiding needless
|
8
|
+
block.
|
9
|
+
|
10
|
+
Before we go into detail, here's who can upgrade without changing anything,
|
11
|
+
and who should make a few adjustments in their code:
|
12
|
+
|
13
|
+
* If you're only using rest-more, e.g. `RC::Facebook` or `RC::Twitter`, etc.,
|
14
|
+
you don't have to change anything. This won't affect rest-more users.
|
15
|
+
(except that JsonDecode is renamed to JsonResponse, and json_decode is
|
16
|
+
renamed to json_response.)
|
17
|
+
|
18
|
+
* If you're only using rest-core's built in middlewares to build your own
|
19
|
+
clients, you don't have to change anything as well. All the hard works are
|
20
|
+
done in rest-core. (except that ErrorHandler works a bit differently now.
|
21
|
+
We'll talk about detail later.)
|
22
|
+
|
23
|
+
* If you're building your own middlewares, then you are the ones who need to
|
24
|
+
make changes. `RC::ASYNC` is changed to a flag to mean whether the callback
|
25
|
+
should be called directly, or only after resuming from the future (fiber
|
26
|
+
or thread). And now you have always to get the response from `yield`, that
|
27
|
+
is, you're forced to pass a callback to `call`.
|
28
|
+
|
29
|
+
This might be a bit user unfriendly at first glimpse, but it would much
|
30
|
+
simplify the internal structure of rest-core, because in the middlewares,
|
31
|
+
you don't have to worry if the user would pass a callback or not, branching
|
32
|
+
everywhere to make it work both synchronously and asynchronously.
|
33
|
+
|
34
|
+
Also, the old fiber based asynchronous HTTP client is removed, in favor
|
35
|
+
of the new _future_ based approach. The new one is more like a superset
|
36
|
+
of the old one, which have anything the old one can provide. Yet internally
|
37
|
+
it works a lot differently. They are both synchronous to the outsides,
|
38
|
+
but while the old one is also synchronous inside, the new one is
|
39
|
+
asynchronous inside, just like the purely asynchronous HTTP client.
|
40
|
+
|
41
|
+
That is, internally, it's always asynchronously, and fiber/async didn't
|
42
|
+
make much difference here now. This is also the reason why I removed
|
43
|
+
the old fiber one. This would make the middleware implementation much
|
44
|
+
easier, considering much fewer possible cases.
|
45
|
+
|
46
|
+
If you don't really understand what above does mean, then just remember,
|
47
|
+
now we ask all middlewares work asynchronously. You have always to work
|
48
|
+
with callbacks which passed along in `app.call(env){ |response| }`
|
49
|
+
That's it.
|
50
|
+
|
51
|
+
So what's the most important improvement? From now on, we have only two
|
52
|
+
modes. One is callback mode, in which case `env[ASYNC]` would be set, and
|
53
|
+
the callback would be called. No exception would be raised in this case.
|
54
|
+
If there's an exception, then it would be passed to the block instead.
|
55
|
+
|
56
|
+
The other mode, which is synchronous, is achieved by the futures. We have
|
57
|
+
two different kinds of futures for now, one is thread based, and the other
|
58
|
+
is fiber based. For RestClient, thread based future would be used. For
|
59
|
+
EventMachine, depending on the context, if the future is created on the
|
60
|
+
main thread, then it would assume it's also wrapped inside a fiber. Since,
|
61
|
+
we can never block the event loop! If you're not calling it in a thread,
|
62
|
+
you must call it in a fiber. But if you're calling it in a thread, then
|
63
|
+
the thread based future would be picked. This is because otherwise it won't
|
64
|
+
work well exchanging information around threads and fibers.
|
65
|
+
|
66
|
+
In short, rest-core would run concurrently in all contexts, archived by
|
67
|
+
either threads or fibers depending on the context, and it would pick the
|
68
|
+
right strategy for you.
|
69
|
+
|
70
|
+
You can see [use-cases.rb][] for all possible use cases.
|
71
|
+
|
72
|
+
It's a bit outdated, but you can also checkout my blog post.
|
73
|
+
[rest-core 2.0 roadmap, thunk based response][post]
|
74
|
+
(p.s. now thunk is renamed to future)
|
75
|
+
|
76
|
+
[use-cases.rb]: https://github.com/cardinalblue/rest-core/blob/master/example/use-cases.rb
|
77
|
+
[post]: http://blogger.godfat.org/2012/06/rest-core-20-roadmap-thunk-based.html
|
78
|
+
|
79
|
+
### Incompatible changes
|
80
|
+
|
81
|
+
* [JsonDecode] is renamed to JsonResponse, and json_decode is also renamed
|
82
|
+
to json_response.
|
83
|
+
* [Json] Now you can use `Json.decode` and `Json.encode` to parse and
|
84
|
+
generate JSONs, instead of `JsonDecode.json_decode`.
|
85
|
+
* [Cache] Support for "cache.post" is removed.
|
86
|
+
* [Cache] The cache key is changed accordingly to support cache for headers
|
87
|
+
and HTTP status. If you don't have persistent cache, this doesn't matter.
|
88
|
+
|
89
|
+
* [EmHttpRequestFiber] is removed in favor of `EmHttpRequest`
|
90
|
+
* cool.io support is removed.
|
91
|
+
* You must provide a block to `app.call(env){ ... }`.
|
92
|
+
* Rename Wrapper#default_app to Wrapper#default_engine
|
93
|
+
|
94
|
+
### Enhancement
|
95
|
+
|
96
|
+
* The default engine is changed from `RestClient` to `Auto`, which would
|
97
|
+
be using `EmHttpRequest` under the context of a event loop, while
|
98
|
+
use `RestClient` in other context as before.
|
99
|
+
|
100
|
+
* `RestCore.eagerload` is introduced to load all constants eagerly. You can
|
101
|
+
use this before loading the application to avoid thread-safety issue in
|
102
|
+
autoload. For the lazies.
|
103
|
+
|
104
|
+
* [JsonResponse] This is originally JsonDecode, and now we prefer multi_json
|
105
|
+
first, yajl-ruby second, lastly json.
|
106
|
+
* [JsonResponse] give JsonResponse a default header Accept: application/json,
|
107
|
+
thanks @ayamomiji
|
108
|
+
* [JsonRequest] This middleware would encode your payload into a JSON.
|
109
|
+
* [CommonLogger] Now we log the request method as well.
|
110
|
+
* [DefaultPayload] Accept arbitrary payload.
|
111
|
+
* [DefaultQuery] Now before merging queries, converting every single key into
|
112
|
+
a string. This allows you to use :symbols for default query.
|
113
|
+
|
114
|
+
* [ErrorHandler] So now ErrorHandler is working differently. It would first
|
115
|
+
try to see if `env[FAIL]` has any exception in it. If there is, then raise
|
116
|
+
it. Otherwise it would call error_handler and expect it to generate an
|
117
|
+
error object. If the error object is an exception, then raise it. If it's
|
118
|
+
not, then it would merge it into `env[FAIL]`. On the other hand, in the
|
119
|
+
case of using callbacks instead of futures, it would pass the exception
|
120
|
+
as the `env[RESPONSE_BODY]` instead. The reason is that you can't raise
|
121
|
+
an exception asynchronously and handle it safely.
|
122
|
+
|
123
|
+
* [Cache] Now response headers and HTTP status are also cached.
|
124
|
+
* [Cache] Not only GET requests are cached, HEAD and OPTIONS are cached too.
|
125
|
+
* [Cache] The cache key is also respecting the request headers too. Suppose
|
126
|
+
you're making a request with different Accept header.
|
127
|
+
|
128
|
+
* [Client] Add Client#wait which would block until all requests for this
|
129
|
+
particular client are done.
|
130
|
+
|
131
|
+
### Bugs fixes
|
132
|
+
|
133
|
+
* [Middleware] Sort the query before generating the request URI, making
|
134
|
+
sure the order is always the same.
|
135
|
+
* [Middleware] The middleware could have no members at all.
|
136
|
+
* [ParseQuery] The fallback function for the absence of Rack is fixed.
|
137
|
+
* [Auto] Only use EmHttpRequest if em-http-request is loaded,
|
138
|
+
thanks @ayamomiji
|
139
|
+
|
3
140
|
## rest-core 1.0.3 -- 2012-08-15
|
4
141
|
|
5
142
|
### Enhancement
|
data/Gemfile
CHANGED
data/README.md
CHANGED
@@ -6,8 +6,6 @@ Lin Jen-Shin ([godfat][]) had given a talk about rest-core on
|
|
6
6
|
[RubyConf Taiwan 2011][talk]. The slide is in English, but the
|
7
7
|
talk is in Mandarin.
|
8
8
|
|
9
|
-
You can also read some other topics at [doc](https://github.com/cardinalblue/rest-core/blob/master/doc/ToC.md).
|
10
|
-
|
11
9
|
[godfat]: https://github.com/godfat
|
12
10
|
[talk]: http://rubyconf.tw/2011/#6
|
13
11
|
|
@@ -24,47 +22,45 @@ Modular Ruby clients interface for REST APIs
|
|
24
22
|
|
25
23
|
There has been an explosion in the number of REST APIs available today.
|
26
24
|
To address the need for a way to access these APIs easily and elegantly,
|
27
|
-
we have developed
|
25
|
+
we have developed rest-core, which consists of composable middleware
|
28
26
|
that allows you to build a REST client for any REST API. Or in the case of
|
29
27
|
common APIs such as Facebook, Github, and Twitter, you can simply use the
|
30
28
|
dedicated clients provided by [rest-more][].
|
31
29
|
|
32
|
-
[rest-core]: https://github.com/cardinalblue/rest-core
|
33
30
|
[rest-more]: https://github.com/cardinalblue/rest-more
|
34
31
|
|
35
32
|
## FEATURES:
|
36
33
|
|
37
34
|
* Modular interface for REST clients similar to WSGI/Rack for servers.
|
38
|
-
*
|
35
|
+
* Concurrent requests with synchronous or asynchronous interfaces with
|
36
|
+
fibers or threads are both supported.
|
39
37
|
|
40
38
|
## REQUIREMENTS:
|
41
39
|
|
42
40
|
### Mandatory:
|
43
41
|
|
44
|
-
* MRI (official CRuby) 1.
|
42
|
+
* MRI (official CRuby) 1.9.3, Rubinius 1.9 and JRuby 1.9
|
45
43
|
* gem rest-client
|
46
44
|
|
47
45
|
### Optional:
|
48
46
|
|
49
|
-
* Fibers only work on Ruby 1.9+
|
50
47
|
* gem [em-http-request][] (if using eventmachine)
|
51
|
-
* gem
|
52
|
-
|
48
|
+
* gem json or yajl-ruby, or multi_json (if `JsonResponse` or
|
49
|
+
`JsonRequest` middlewares are used)
|
53
50
|
|
54
51
|
[em-http-request]: https://github.com/igrigorik/em-http-request
|
55
|
-
[cool.io-http]: https://github.com/godfat/cool.io-http
|
56
52
|
|
57
53
|
## INSTALLATION:
|
58
54
|
|
59
55
|
``` shell
|
60
|
-
|
56
|
+
gem install rest-core
|
61
57
|
```
|
62
58
|
|
63
59
|
Or if you want development version, put this in Gemfile:
|
64
60
|
|
65
61
|
``` ruby
|
66
|
-
|
67
|
-
|
62
|
+
gem 'rest-core', :git => 'git://github.com/cardinalblue/rest-core.git',
|
63
|
+
:submodules => true
|
68
64
|
```
|
69
65
|
|
70
66
|
If you just want to use Facebook or Twitter clients, please take a look at
|
@@ -74,235 +70,231 @@ If you just want to use Facebook or Twitter clients, please take a look at
|
|
74
70
|
|
75
71
|
## Build Your Own Clients:
|
76
72
|
|
77
|
-
You can use `RestCore::Builder` to build your own dedicated
|
73
|
+
You can use `RestCore::Builder` to build your own dedicated clients.
|
74
|
+
Note that `RC` is an alias of `RestCore`
|
78
75
|
|
79
76
|
``` ruby
|
80
|
-
|
81
|
-
|
82
|
-
|
83
|
-
|
84
|
-
|
85
|
-
|
86
|
-
|
87
|
-
use s::Cache , nil, 3600
|
88
|
-
run s::RestClient # the simplest and easier HTTP client
|
89
|
-
end
|
77
|
+
require 'rest-core'
|
78
|
+
YourClient = RC::Builder.client do
|
79
|
+
use RC::DefaultSite , 'https://api.github.com/users/'
|
80
|
+
use RC::JsonResponse, true
|
81
|
+
use RC::CommonLogger, method(:puts)
|
82
|
+
use RC::Cache , nil, 3600
|
83
|
+
end
|
90
84
|
```
|
91
85
|
|
92
86
|
And use it with per-instance basis (clients could have different
|
93
87
|
configuration, e.g. different cache time or timeout time):
|
94
88
|
|
95
89
|
``` ruby
|
96
|
-
|
97
|
-
|
98
|
-
|
90
|
+
client = YourClient.new(:cache => {})
|
91
|
+
client.get('cardinalblue') # cache miss
|
92
|
+
client.get('cardinalblue') # cache hit
|
99
93
|
|
100
|
-
|
101
|
-
|
102
|
-
|
94
|
+
client.site = 'http://github.com/api/v2/json/user/show/'
|
95
|
+
client.get('cardinalblue') # cache miss
|
96
|
+
client.get('cardinalblue') # cache hit
|
103
97
|
```
|
104
98
|
|
105
|
-
|
106
|
-
|
107
|
-
for concepts.
|
108
|
-
|
109
|
-
[example/rest-client.rb]: https://github.com/cardinalblue/rest-core/blob/master/example/rest-client.rb
|
110
|
-
[rest-more]: https://github.com/cardinalblue/rest-more
|
111
|
-
[slides]: http://www.godfat.org/slide/2011-08-27-rest-core.html
|
112
|
-
[rubyconf.tw]: http://rubyconf.tw/2011/#6
|
113
|
-
|
114
|
-
## Asynchronous HTTP Requests:
|
115
|
-
|
116
|
-
I/O bound operations shouldn't be blocking the CPU! If you have a reactor,
|
117
|
-
i.e. event loop, you should take the advantage of that to make HTTP requests
|
118
|
-
not block the whole process/thread. For now, we support eventmachine and
|
119
|
-
cool.io. Below is an example for eventmachine:
|
99
|
+
You can also make concurrent requests easily:
|
100
|
+
(see "Advanced Concurrent HTTP Requests -- Embrace the Future" for detail)
|
120
101
|
|
121
102
|
``` ruby
|
122
|
-
|
123
|
-
|
124
|
-
|
125
|
-
|
126
|
-
use s::DefaultSite , 'https://api.github.com/users/'
|
127
|
-
use s::JsonDecode , true
|
128
|
-
use s::CommonLogger, method(:puts)
|
129
|
-
use s::Cache , nil, 3600
|
130
|
-
run s::EmHttpRequest
|
131
|
-
end
|
103
|
+
a = [client.get('cardinalblue')['name'], client.get('godfat')['name']]
|
104
|
+
puts "It's not blocking... but doing concurrent requests underneath"
|
105
|
+
p a # here we want the values, so it blocks here
|
106
|
+
puts "DONE"
|
132
107
|
```
|
133
108
|
|
134
|
-
|
135
|
-
available. That is the block is the callback for the request.
|
109
|
+
Callback mode also available:
|
136
110
|
|
137
111
|
``` ruby
|
138
|
-
|
139
|
-
|
140
|
-
|
141
|
-
|
142
|
-
EM.stop
|
143
|
-
}
|
144
|
-
puts "It's not blocking..."
|
145
|
-
}
|
112
|
+
client.get('cardinalblue'){ |v| p v }
|
113
|
+
puts "It's not blocking... but doing concurrent requests underneath"
|
114
|
+
client.wait # we block here to wait for the request done
|
115
|
+
puts "DONE"
|
146
116
|
```
|
147
117
|
|
148
|
-
|
149
|
-
|
150
|
-
|
151
|
-
If you don't understand what does this mean, you can take a look at
|
152
|
-
[em-synchrony][]. It's basically the same idea.
|
118
|
+
Runnable example is at: [example/simple.rb][]. Please see [rest-more][]
|
119
|
+
for more complex examples to build clients, and [slides][] from
|
120
|
+
[rubyconf.tw/2011][rubyconf.tw] for concepts.
|
153
121
|
|
154
|
-
|
155
|
-
|
156
|
-
|
157
|
-
|
158
|
-
EM.stop
|
159
|
-
}.resume
|
160
|
-
puts "It's not blocking..."
|
161
|
-
}
|
162
|
-
```
|
163
|
-
|
164
|
-
[em-synchrony]: https://github.com/igrigorik/em-synchrony
|
165
|
-
|
166
|
-
Runnable example is here: [example/eventmachine.rb][].
|
167
|
-
You can also make multi-requests synchronously like this:
|
168
|
-
|
169
|
-
``` ruby
|
170
|
-
EM.run{
|
171
|
-
Fiber.new{
|
172
|
-
fiber = Fiber.current
|
173
|
-
result = {}
|
174
|
-
client.get('cardinalblue'){ |response|
|
175
|
-
result[0] = response
|
176
|
-
fiber.resume(result) if result.size == 2
|
177
|
-
}
|
178
|
-
client.get('cardinalblue'){ |response|
|
179
|
-
result[1] = response
|
180
|
-
fiber.resume(result) if result.size == 2
|
181
|
-
}
|
182
|
-
p Fiber.yield
|
183
|
-
EM.stop
|
184
|
-
}.resume
|
185
|
-
puts "It's not blocking..."
|
186
|
-
}
|
187
|
-
```
|
188
|
-
|
189
|
-
Runnable example is here: [example/multi.rb][].
|
190
|
-
|
191
|
-
[example/eventmachine.rb]: https://github.com/cardinalblue/rest-core/blob/master/example/eventmachine.rb
|
192
|
-
[example/multi.rb]: https://github.com/cardinalblue/rest-core/blob/master/example/multi.rb
|
193
|
-
|
194
|
-
## Supported HTTP clients:
|
122
|
+
[example/simple.rb]: https://github.com/cardinalblue/rest-core/blob/master/example/simple.rb
|
123
|
+
[rest-more]: https://github.com/cardinalblue/rest-more
|
124
|
+
[slides]: http://www.godfat.org/slide/2011-08-27-rest-core.html
|
125
|
+
[rubyconf.tw]: http://rubyconf.tw/2011/#6
|
195
126
|
|
196
|
-
|
197
|
-
|
198
|
-
* `
|
199
|
-
* `
|
200
|
-
|
127
|
+
## List of built-in Middlewares:
|
128
|
+
|
129
|
+
* `RC::AuthBasic`
|
130
|
+
* `RC::Bypass`
|
131
|
+
* `RC::Cache`
|
132
|
+
* `RC::CommonLogger`
|
133
|
+
* `RC::DefaultHeaders`
|
134
|
+
* `RC::DefaultPayload`
|
135
|
+
* `RC::DefaultQuery`
|
136
|
+
* `RC::DefaultSite`
|
137
|
+
* `RC::Defaults`
|
138
|
+
* `RC::ErrorDetector`
|
139
|
+
* `RC::ErrorDetectorHttp`
|
140
|
+
* `RC::ErrorHandler`
|
141
|
+
* `RC::FollowRedirect`
|
142
|
+
* `RC::JsonRequest`
|
143
|
+
* `RC::JsonResponse`
|
144
|
+
* `RC::Oauth1Header`
|
145
|
+
* `RC::Oauth2Header`
|
146
|
+
* `RC::Oauth2Query`
|
147
|
+
* `RC::Timeout`
|
201
148
|
|
202
149
|
## Build Your Own Middlewares:
|
203
150
|
|
204
151
|
To be added.
|
205
152
|
|
206
|
-
##
|
207
|
-
|
208
|
-
To be added.
|
209
|
-
|
210
|
-
## rest-core users:
|
211
|
-
|
212
|
-
* [topcoder](https://github.com/miaout17/topcoder)
|
213
|
-
* [s2sync](https://github.com/brucehsu/s2sync)
|
214
|
-
* [s2sync_web](https://github.com/brucehsu/s2sync_web)
|
153
|
+
## Advanced Concurrent HTTP Requests -- Embrace the Future
|
215
154
|
|
216
|
-
|
155
|
+
### The Interface
|
217
156
|
|
218
|
-
|
157
|
+
There are a number of different ways to make concurrent requests in
|
158
|
+
rest-core. They could be roughly categorized to two different forms.
|
159
|
+
One is using the well known callbacks, while the other one is using
|
160
|
+
through a technique called [future][]. Basically, it means it would
|
161
|
+
return you a promise, which would eventually become the real value
|
162
|
+
(response here) you were asking for whenever you really want it.
|
163
|
+
Otherwise, the program keeps running until the value is evaluated,
|
164
|
+
and blocks there if the computation (response) hasn't been done yet.
|
165
|
+
If the computation is already done, then it would simply return you
|
166
|
+
the result.
|
219
167
|
|
220
|
-
|
168
|
+
Here's a very simple example for using futures:
|
221
169
|
|
222
|
-
|
170
|
+
``` ruby
|
171
|
+
require 'rest-core'
|
172
|
+
YourClient = RC::Builder.client do
|
173
|
+
use RC::DefaultSite , 'https://api.github.com/users/'
|
174
|
+
use RC::JsonResponse, true
|
175
|
+
use RC::CommonLogger, method(:puts)
|
176
|
+
end
|
177
|
+
|
178
|
+
client = YourClient.new
|
179
|
+
puts "rest-client with threads doing concurrent requests"
|
180
|
+
a = [client.get('cardinalblue')['name'], client.get('godfat')['name']]
|
181
|
+
puts "It's not blocking... but doing concurrent requests underneath"
|
182
|
+
p a # here we want the values, so it blocks here
|
183
|
+
puts "DONE"
|
184
|
+
```
|
223
185
|
|
224
|
-
|
186
|
+
And here's a corresponded version for using callbacks:
|
225
187
|
|
226
|
-
|
227
|
-
|
188
|
+
``` ruby
|
189
|
+
require 'rest-core'
|
190
|
+
YourClient = RC::Builder.client do
|
191
|
+
use RC::DefaultSite , 'https://api.github.com/users/'
|
192
|
+
use RC::JsonResponse, true
|
193
|
+
use RC::CommonLogger, method(:puts)
|
194
|
+
end
|
195
|
+
|
196
|
+
client = YourClient.new
|
197
|
+
puts "rest-client with threads doing concurrent requests"
|
198
|
+
client.get('cardinalblue'){ |v|
|
199
|
+
p v['name']
|
200
|
+
}.
|
201
|
+
get('godfat'){ |v|
|
202
|
+
p v['name']
|
203
|
+
}
|
204
|
+
puts "It's not blocking... but doing concurrent requests underneath"
|
205
|
+
client.wait # until all requests are done
|
206
|
+
puts "DONE"
|
207
|
+
```
|
228
208
|
|
229
|
-
|
230
|
-
For instance, `RestCore::RestClient` is an HTTP client which uses
|
231
|
-
rest-client gem (`::RestClient`) to make HTTP requests.
|
209
|
+
You can pick whatever works for you.
|
232
210
|
|
233
|
-
|
234
|
-
For instance, `RestCore::DefaultSite` is a middleware which would add
|
235
|
-
default site URL in front of the request URI if it is not started with
|
236
|
-
http://, thus you can do this: `RestCore::Facebook.get('4')` without
|
237
|
-
specifying where the site (Facebook) it is.
|
211
|
+
[future]: http://en.wikipedia.org/wiki/Futures_and_promises
|
238
212
|
|
239
|
-
|
240
|
-
middlewares into another middleware. Currently, it's used in
|
241
|
-
`RestCore::Buidler` and `RestCore::Cache`.
|
213
|
+
### What Concurrency Model to Choose?
|
242
214
|
|
243
|
-
|
244
|
-
|
215
|
+
In the above example, we're using rest-client with threads, which works
|
216
|
+
for most of cases. But you might also want to use em-http-request with
|
217
|
+
EventMachine, which is using a faster HTTP parser. In theory, it should
|
218
|
+
be much more efficient than rest-client and threads.
|
245
219
|
|
246
|
-
|
247
|
-
|
220
|
+
To pick em-http-request, you must run the requests inside the EventMachine's
|
221
|
+
event loop, and also wrap your request with either a thread or a fiber,
|
222
|
+
because we can't block the event loop and ask em-http-request to finish
|
223
|
+
its job making requests.
|
248
224
|
|
249
|
-
|
250
|
-
_client_ by `RestCore::Builder`. It contains a number of convenient
|
251
|
-
functions which is generally useful.
|
225
|
+
Here's an example of using em-http-request with threads:
|
252
226
|
|
253
|
-
|
254
|
-
|
255
|
-
|
227
|
+
``` ruby
|
228
|
+
require 'em-http-request'
|
229
|
+
require 'rest-core'
|
230
|
+
YourClient = RC::Builder.client do
|
231
|
+
use RC::DefaultSite , 'https://api.github.com/users/'
|
232
|
+
use RC::JsonResponse, true
|
233
|
+
use RC::CommonLogger, method(:puts)
|
234
|
+
end
|
235
|
+
|
236
|
+
client = YourClient.new
|
237
|
+
puts "eventmachine with threads doing concurrent requests"
|
238
|
+
EM.run{
|
239
|
+
Thread.new{
|
240
|
+
p [client.get('cardinalblue')['name'], client.get('godfat')['name']]
|
241
|
+
puts "DONE"
|
242
|
+
EM.stop
|
243
|
+
}
|
244
|
+
puts "It's not blocking... but doing concurrent requests underneath"
|
245
|
+
}
|
246
|
+
```
|
256
247
|
|
257
|
-
|
258
|
-
response. It's mostly seen in `@app.call(env)` See other explanation
|
259
|
-
such as `env[RestCore::REQUEST_METHOD]` for more detail.
|
248
|
+
And here's an example of using em-http-request with fibers:
|
260
249
|
|
261
|
-
|
262
|
-
|
263
|
-
|
250
|
+
``` ruby
|
251
|
+
require 'fiber' # remember to require fiber first,
|
252
|
+
require 'em-http-request' # or rest-core won't pick fibers
|
253
|
+
require 'rest-core'
|
254
|
+
YourClient = RC::Builder.client do
|
255
|
+
use RC::DefaultSite , 'https://api.github.com/users/'
|
256
|
+
use RC::JsonResponse, true
|
257
|
+
use RC::CommonLogger, method(:puts)
|
258
|
+
end
|
259
|
+
|
260
|
+
client = YourClient.new
|
261
|
+
puts "eventmachine with fibers doing concurrent requests"
|
262
|
+
EM.run{
|
263
|
+
Fiber.new{
|
264
|
+
p [client.get('cardinalblue')['name'], client.get('godfat')['name']]
|
265
|
+
puts "DONE"
|
266
|
+
EM.stop
|
267
|
+
}
|
268
|
+
puts "It's not blocking... but doing concurrent requests underneath"
|
269
|
+
}
|
270
|
+
```
|
264
271
|
|
265
|
-
|
266
|
-
|
267
|
-
|
268
|
-
simply `"4"`. In the case of built-in Facebook client, the
|
269
|
-
`RestCore::DefaultSite` middleware would take care of the site.
|
272
|
+
As you can see, both of them are quite similar to each other, because the
|
273
|
+
idea behind the scene is the same. If you don't know what concurrency model
|
274
|
+
to pick, start with rest-client since it's the easiest one to setup.
|
270
275
|
|
271
|
-
|
272
|
-
|
273
|
-
|
274
|
-
|
276
|
+
A full runnable example is at: [example/multi.rb][]. If you want to know
|
277
|
+
all the possible use cases, you can also see: [example/use-cases.rb][]. It's
|
278
|
+
also served as a test for each possible combinations, so it's quite complex
|
279
|
+
and complete.
|
275
280
|
|
276
|
-
|
277
|
-
values are payload values. Both keys and values' type should be String,
|
278
|
-
not Symbol. Values with nil or false would be ignored. Both keys and values
|
279
|
-
would be escaped automatically.
|
281
|
+
[example/multi.rb]: https://github.com/cardinalblue/rest-core/blob/master/example/multi.rb
|
280
282
|
|
281
|
-
|
282
|
-
values are header values. Both keys and values' type should be String,
|
283
|
-
not Symbol. Values with nil or false would be ignored.
|
283
|
+
[example/use-cases.rb]: https://github.com/cardinalblue/rest-core/blob/master/example/use-cases.rb
|
284
284
|
|
285
|
-
|
286
|
-
Might be nil if there's no response or not yet making HTTP request.
|
285
|
+
## rest-core users:
|
287
286
|
|
288
|
-
*
|
289
|
-
|
290
|
-
|
287
|
+
* [topcoder](https://github.com/miaout17/topcoder)
|
288
|
+
* [s2sync](https://github.com/brucehsu/s2sync)
|
289
|
+
* [s2sync_web](https://github.com/brucehsu/s2sync_web)
|
291
290
|
|
292
|
-
|
293
|
-
for the response headers. Both keys and values' type should be String.
|
291
|
+
## Powered sites:
|
294
292
|
|
295
|
-
*
|
296
|
-
indicates that if we're only asking for modified `env`, instead of making
|
297
|
-
real requests. It's used to ask for the real request URI, etc.
|
293
|
+
* [PicCollage](http://pic-collage.com/)
|
298
294
|
|
299
|
-
|
300
|
-
could be any objects, it's handled by `RestCore::ErrorDetector` or any
|
301
|
-
other custom _middleware_.
|
295
|
+
## CHANGES:
|
302
296
|
|
303
|
-
*
|
304
|
-
could be any objects, it's handled by `RestCore::CommonLogger` or
|
305
|
-
any other custom _middleware_.
|
297
|
+
* [CHANGES](https://github.com/cardinalblue/rest-core/blob/master/CHANGES.md)
|
306
298
|
|
307
299
|
## CONTRIBUTORS:
|
308
300
|
|