hurley-http-cache 0.1.0.beta
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- data/LICENSE +13 -0
- data/README.md +93 -0
- data/lib/hurley-http-cache.rb +1 -0
- data/lib/hurley/http_cache.rb +242 -0
- data/lib/hurley/http_cache/cache_control.rb +118 -0
- data/lib/hurley/http_cache/request.rb +51 -0
- data/lib/hurley/http_cache/response.rb +186 -0
- data/lib/hurley/http_cache/storage.rb +182 -0
- data/test/http_cache/cache_control_test.rb +79 -0
- data/test/http_cache/request_test.rb +74 -0
- data/test/http_cache/response_test.rb +116 -0
- data/test/http_cache/storage_test.rb +80 -0
- data/test/live_http_cache_test.rb +238 -0
- data/test/support/server.rb +103 -0
- data/test/test_helper.rb +23 -0
- metadata +114 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: fb71f71333adf5e4e068bb81c474c092cc16a5f4
|
4
|
+
data.tar.gz: 55b436e2743064185ac7383b3e9ec71389dc0950
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: ee330c420cb6080c33e2ecb673a1de0efc99e1498c413cac9861d4555e6c30c12fb056fa1a7c67dca2b49047d872860599c14c07a32a951467187b952c88940a
|
7
|
+
data.tar.gz: 19de7bdbe98a40d5fbc70f56c3e61c721f1a0cbcdf6c9c49ef9b3d040c6592492f80fe40b297c172211c6aba4453291030882912fe9ab62115590dae0930599b
|
data/LICENSE
ADDED
@@ -0,0 +1,13 @@
|
|
1
|
+
Copyright 2015 Plataformatec.
|
2
|
+
|
3
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
4
|
+
you may not use this file except in compliance with the License.
|
5
|
+
You may obtain a copy of the License at
|
6
|
+
|
7
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
8
|
+
|
9
|
+
Unless required by applicable law or agreed to in writing, software
|
10
|
+
distributed under the License is distributed on an "AS IS" BASIS,
|
11
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
12
|
+
See the License for the specific language governing permissions and
|
13
|
+
limitations under the License.
|
data/README.md
ADDED
@@ -0,0 +1,93 @@
|
|
1
|
+
# Hurley Http Cache
|
2
|
+
|
3
|
+
[![Build Status](https://secure.travis-ci.org/plataformatec/hurley-http-cache.png?branch=master)](https://travis-ci.org/plataformatec/hurley-http-cache)
|
4
|
+
|
5
|
+
a [Hurley](https://github.com/lostisland/hurley) connection that respects HTTP cache,
|
6
|
+
by checking expiration and validation of the stored responses. This gem is a direct
|
7
|
+
reimplementation of the [faraday-http-cache](https://github.com/plataformatec/faraday-http-cache)
|
8
|
+
gem.
|
9
|
+
|
10
|
+
## Installation
|
11
|
+
|
12
|
+
Add it to your Gemfile:
|
13
|
+
|
14
|
+
```ruby
|
15
|
+
gem 'hurley-http-cache'
|
16
|
+
```
|
17
|
+
|
18
|
+
## Usage and configuration
|
19
|
+
|
20
|
+
You can use an instance of the `Hurley::HttpCache` as the connection for your
|
21
|
+
`hurley` client.
|
22
|
+
|
23
|
+
```ruby
|
24
|
+
require 'hurley'
|
25
|
+
require 'hurley/http_cache'
|
26
|
+
|
27
|
+
client = Hurley::Client.new
|
28
|
+
client.connection = Hurley::HttpCache.new
|
29
|
+
```
|
30
|
+
|
31
|
+
The middleware accepts a `store` option for the cache backend responsible for
|
32
|
+
recording the API responses that should be stored. Stores should respond to
|
33
|
+
`write` and `read`, just like an object from the `ActiveSupport::Cache` API.
|
34
|
+
|
35
|
+
```ruby
|
36
|
+
store = ActiveSupport::Cache.lookup_store(:mem_cache_store, ['localhost:11211'])
|
37
|
+
|
38
|
+
client = Hurley::Client.new
|
39
|
+
# Use the connection with a Memcache server.
|
40
|
+
client.connection = Hurley::HttpCache.new(store: store)
|
41
|
+
|
42
|
+
# Or use the Rails.cache instance inside your Rails app.
|
43
|
+
client.connection = Hurley::HttpCache.new(store: Rails.cache)
|
44
|
+
```
|
45
|
+
|
46
|
+
By default, the `Hurley::HttpCache` connection will use the `Hurley.default_connection`
|
47
|
+
to perform the real HTTP requests when we can't use a cached response. If you
|
48
|
+
want to use a different connection object, just pass it when creating the
|
49
|
+
`Hurley::HttpCache` object.
|
50
|
+
|
51
|
+
```ruby
|
52
|
+
require 'hurley'
|
53
|
+
require 'hurley-excon'
|
54
|
+
|
55
|
+
client = Hurley.new
|
56
|
+
client.connection = Hurley::HttpCache.new(HurleyExcon::Connection.new)
|
57
|
+
```
|
58
|
+
|
59
|
+
The default store provided is a simple in memory cache that lives on the client
|
60
|
+
instance. This type of store **might not be persisted across multiple processes
|
61
|
+
or connection instances** so it is probably not suitable for most production
|
62
|
+
environments. Make sure that you configure a store that is suitable for you.
|
63
|
+
|
64
|
+
the stdlib `JSON` module is used for serialization by default.
|
65
|
+
If you expect to be dealing with images, you can use [Marshal][http://ruby-doc.org//core-2.2.0/Marshal.html]
|
66
|
+
instead, or if you want to use another json library like `oj` or `yajl-ruby`.
|
67
|
+
|
68
|
+
```ruby
|
69
|
+
client = Hurley::Client.new
|
70
|
+
client.connection = Hurley::HttpCache.new(store: Rails.cache, serializer: Marshal)
|
71
|
+
```
|
72
|
+
|
73
|
+
## Logging
|
74
|
+
|
75
|
+
You can provide a `logger` option that will be receive debug informations based
|
76
|
+
connection operations:
|
77
|
+
|
78
|
+
```ruby
|
79
|
+
client = Hurley::Client.new
|
80
|
+
client.connection = Hurley::HttpCache.new(logger: Rails.logger)
|
81
|
+
|
82
|
+
client.get('http://site/api/users')
|
83
|
+
# logs "HTTP Cache: [GET /users] miss, store"
|
84
|
+
```
|
85
|
+
|
86
|
+
## See it live
|
87
|
+
|
88
|
+
You can clone this repository, install it's dependencies with Bundler (run `bundle install`) and
|
89
|
+
execute the files under the `examples` directory to see a sample of the gem usage.
|
90
|
+
|
91
|
+
## License
|
92
|
+
|
93
|
+
Copyright (c) 2015 Plataformatec. See LICENSE file.
|
@@ -0,0 +1 @@
|
|
1
|
+
require 'hurley/http_cache'
|
@@ -0,0 +1,242 @@
|
|
1
|
+
require 'hurley'
|
2
|
+
require 'hurley/http_cache/cache_control'
|
3
|
+
require 'hurley/http_cache/request'
|
4
|
+
require 'hurley/http_cache/response'
|
5
|
+
require 'hurley/http_cache/storage'
|
6
|
+
|
7
|
+
module Hurley
|
8
|
+
class HttpCache
|
9
|
+
UNSAFE_VERBS = [:post, :put, :delete, :patch]
|
10
|
+
|
11
|
+
ERROR_STATUSES = 400..499
|
12
|
+
|
13
|
+
# Public: Initialize a Hurley connection that supports HTTP caching.
|
14
|
+
#
|
15
|
+
# connection - A real connection object that can make HTTP requests
|
16
|
+
# (default: the Hurley.default_connection value).
|
17
|
+
# shared_cache - A flag to indicate if the connection should act as a shared
|
18
|
+
# cache or not (default: true)
|
19
|
+
# serializer - A serializer object for the cache store (default: nil).
|
20
|
+
# store - A cache store to receive the stored requests and responses
|
21
|
+
# (default: nil).
|
22
|
+
# logger - A Logger object (default: nil).
|
23
|
+
#
|
24
|
+
# Examples:
|
25
|
+
#
|
26
|
+
# # Initialize the connection with the Rails logger.
|
27
|
+
# client = Hurley.new
|
28
|
+
# client.connection = Hurley::HttpCache.new(logger: Rails.logger)
|
29
|
+
#
|
30
|
+
# # Initialize the connection with the Hurley Excon connection adapter.
|
31
|
+
# require 'hurley-excon'
|
32
|
+
#
|
33
|
+
# client = Hurley.new
|
34
|
+
# client.connection = Hurley::HttpCache.new(HurleyExcon::Connection.new)
|
35
|
+
def initialize(connection = nil, shared_cache: true, serializer: nil, store: nil, logger: nil)
|
36
|
+
@connection = connection || Hurley.default_connection
|
37
|
+
@logger = logger
|
38
|
+
@shared_cache = shared_cache
|
39
|
+
|
40
|
+
@storage = Storage.new(store: store, serializer: serializer, logger: logger)
|
41
|
+
end
|
42
|
+
|
43
|
+
# Public: Process the client request to try to serve a cache response.
|
44
|
+
# On a cacheable request, we will attempt to locate a valid stored response
|
45
|
+
# to serve. On a cache miss, we will forward the request and try to store
|
46
|
+
# the response for future calls.
|
47
|
+
# If the request can't be cached, the request will be delegated directly
|
48
|
+
# to the underlying connection and does nothing to the response.
|
49
|
+
# The processed steps will be recorded to be logged once the whole
|
50
|
+
# process is finished.
|
51
|
+
#
|
52
|
+
# request - The incoming Hurley::Request object.
|
53
|
+
#
|
54
|
+
# Returns a Hurley::Response object.
|
55
|
+
def call(request)
|
56
|
+
@trace = []
|
57
|
+
request = Hurley::HttpCache::Request.new(request)
|
58
|
+
|
59
|
+
if request.cacheable?
|
60
|
+
if request.no_cache?
|
61
|
+
response = bypass(request)
|
62
|
+
else
|
63
|
+
response = process(request)
|
64
|
+
end
|
65
|
+
else
|
66
|
+
trace :unacceptable
|
67
|
+
response = forward(request)
|
68
|
+
end
|
69
|
+
|
70
|
+
delete(request, response) if should_delete?(response.status_code, request.verb)
|
71
|
+
log_request(request)
|
72
|
+
|
73
|
+
response
|
74
|
+
end
|
75
|
+
|
76
|
+
private
|
77
|
+
|
78
|
+
# Internal: Perform a request bypassing the whole caching mechanism, but
|
79
|
+
# stores the response if possible.
|
80
|
+
#
|
81
|
+
# request - The incoming Hurley::HttpCache::Request object.
|
82
|
+
#
|
83
|
+
# Returns a Hurley::Response object.
|
84
|
+
def bypass(request)
|
85
|
+
trace :bypass
|
86
|
+
|
87
|
+
response = forward(request)
|
88
|
+
store(request, Hurley::HttpCache::Response.new(response))
|
89
|
+
response
|
90
|
+
end
|
91
|
+
|
92
|
+
# Internal: Locate a valid response or forwards the call to the real connection
|
93
|
+
# object. If no entry is present on the storage, a real response will be
|
94
|
+
# retrieved and stored. But if a fresh response already exists in the cache,
|
95
|
+
# we return it back to the client instead of doing a real HTTP request.
|
96
|
+
#
|
97
|
+
# In the case of a existing response that isn't fresh anymore, we will
|
98
|
+
# revalidate the response back with its origin server.
|
99
|
+
#
|
100
|
+
# request - The incoming Hurley::HttpCache::Request object.
|
101
|
+
#
|
102
|
+
# Returns the 'Hurley::HttpCache::Response' object to be served.
|
103
|
+
def process(request)
|
104
|
+
entry = @storage.read(request)
|
105
|
+
|
106
|
+
return fetch(request) if entry.nil?
|
107
|
+
|
108
|
+
entry = Hurley::HttpCache::Response.restore(request.__getobj__, entry)
|
109
|
+
|
110
|
+
if entry.fresh?
|
111
|
+
trace :fresh
|
112
|
+
response = entry
|
113
|
+
else
|
114
|
+
response = validate(entry, request)
|
115
|
+
end
|
116
|
+
|
117
|
+
response
|
118
|
+
end
|
119
|
+
|
120
|
+
# Internal: Fetch a missing response from the real connection object and
|
121
|
+
# stores it in the cache.
|
122
|
+
#
|
123
|
+
# request - The incoming Hurley::HttpCache::Request object.
|
124
|
+
#
|
125
|
+
# Returns the fresh 'Hurley::Response' object.
|
126
|
+
def fetch(request)
|
127
|
+
trace :miss
|
128
|
+
|
129
|
+
response = forward(request)
|
130
|
+
store(request, Hurley::HttpCache::Response.new(response))
|
131
|
+
response
|
132
|
+
end
|
133
|
+
|
134
|
+
# Internal: Validate a stored entry back to it's origin server
|
135
|
+
# using the 'If-Modified-Since' and 'If-None-Match' headers with the
|
136
|
+
# existing 'Last-Modified' and 'ETag' headers. If the new response
|
137
|
+
# is marked as 'Not Modified', the previous stored response will be used
|
138
|
+
# and returned. Otherwise, the freshly new response will be stored
|
139
|
+
# (replacing the old one) and used.
|
140
|
+
#
|
141
|
+
# entry - A stale Hurley::HttpCache::Response retrieved from the cache.
|
142
|
+
# request - The incoming Hurley::HttpCache::Request object.
|
143
|
+
#
|
144
|
+
# Returns the 'Hurley::HttpCache::Response' to be forwarded to the client.
|
145
|
+
def validate(entry, request)
|
146
|
+
header = request.header
|
147
|
+
header['If-Modified-Since'] = entry.last_modified if entry.last_modified
|
148
|
+
header['If-None-Match'] = entry.etag if entry.etag
|
149
|
+
|
150
|
+
response = Hurley::HttpCache::Response.new(forward(request))
|
151
|
+
|
152
|
+
if response.not_modified?
|
153
|
+
trace :valid
|
154
|
+
entry.header.update(response.header)
|
155
|
+
response = entry
|
156
|
+
end
|
157
|
+
store(request, response)
|
158
|
+
response
|
159
|
+
end
|
160
|
+
|
161
|
+
# Internal: Attempt to store the response into the cache.
|
162
|
+
# If the response isn't cacheable we do nothing.
|
163
|
+
#
|
164
|
+
# request - A Hurley::HttpCache::Request object.
|
165
|
+
# response - A Hurley::HttpCache::Response object.
|
166
|
+
#
|
167
|
+
# Returns nothing.
|
168
|
+
def store(request, response)
|
169
|
+
if shared_cache? ? response.cacheable_in_shared_cache? : response.cacheable_in_private_cache?
|
170
|
+
trace :store
|
171
|
+
@storage.write(request, response)
|
172
|
+
else
|
173
|
+
trace :invalid
|
174
|
+
end
|
175
|
+
end
|
176
|
+
|
177
|
+
# Internal: Checks if the current request method should remove any existing
|
178
|
+
# cache entries for the same resource.
|
179
|
+
#
|
180
|
+
# Returns true or false.
|
181
|
+
def should_delete?(status_code, verb)
|
182
|
+
UNSAFE_VERBS.include?(verb) && !ERROR_STATUSES.cover?(status_code)
|
183
|
+
end
|
184
|
+
|
185
|
+
# Internal: Delete an existing entry from the cache, based on the request
|
186
|
+
# URL or any of the 'Location' aware headers of the response.
|
187
|
+
#
|
188
|
+
# request - a Hurley::Request object.
|
189
|
+
# response - a Hurley::Response object.
|
190
|
+
#
|
191
|
+
# Returns nothing.
|
192
|
+
def delete(request, response)
|
193
|
+
trace :delete
|
194
|
+
|
195
|
+
urls = [response.header['Location'], response.header['Content-Location'], request.url]
|
196
|
+
urls.each do |url|
|
197
|
+
@storage.delete(url) if url
|
198
|
+
end
|
199
|
+
end
|
200
|
+
|
201
|
+
# Internal: Log the trace info about the incoming request and how the cache
|
202
|
+
# handled it.
|
203
|
+
# This method does nothing if theresn't a logger present.
|
204
|
+
#
|
205
|
+
# request - The Hurley::Request object that represents the request.
|
206
|
+
#
|
207
|
+
# Returns nothing.
|
208
|
+
def log_request(request)
|
209
|
+
return unless @logger
|
210
|
+
|
211
|
+
@logger.debug do
|
212
|
+
verb = request.verb.to_s.upcase
|
213
|
+
path = request.url.request_uri
|
214
|
+
"HTTP Cache: [#{verb} #{path}] #{@trace.join(', ')}"
|
215
|
+
end
|
216
|
+
end
|
217
|
+
|
218
|
+
# Internal: Calls the original connection object with the current
|
219
|
+
# Hurley::Request object.
|
220
|
+
#
|
221
|
+
# Returns a Hurley::Response object.
|
222
|
+
def forward(request)
|
223
|
+
@connection.call(request.__getobj__)
|
224
|
+
end
|
225
|
+
|
226
|
+
# Internal: Should this connection act like a 'shared cache' according
|
227
|
+
# to the the definition in RFC 2616?
|
228
|
+
def shared_cache?
|
229
|
+
@shared_cache
|
230
|
+
end
|
231
|
+
|
232
|
+
# Internal: Record a traced action to be used by the logger once the
|
233
|
+
# request/response phase is finished.
|
234
|
+
#
|
235
|
+
# operation - the name of the performed action, a String or Symbol.
|
236
|
+
#
|
237
|
+
# Returns nothing.
|
238
|
+
def trace(operation)
|
239
|
+
@trace << operation
|
240
|
+
end
|
241
|
+
end
|
242
|
+
end
|
@@ -0,0 +1,118 @@
|
|
1
|
+
module Hurley
|
2
|
+
class HttpCache
|
3
|
+
# Internal: A class to represent the 'Cache-Control' header options.
|
4
|
+
# This implementation is based on 'rack-cache' internals by Ryan Tomayko.
|
5
|
+
# It breaks the several directives into keys/values and stores them into
|
6
|
+
# a Hash.
|
7
|
+
class CacheControl
|
8
|
+
|
9
|
+
# Internal: Initialize a new CacheControl instance.
|
10
|
+
#
|
11
|
+
# header - The String of the 'Cache-Control' header.
|
12
|
+
def initialize(header)
|
13
|
+
@directives = parse(header.to_s)
|
14
|
+
end
|
15
|
+
|
16
|
+
# Internal: Check if the 'public' directive is present.
|
17
|
+
def public?
|
18
|
+
@directives['public']
|
19
|
+
end
|
20
|
+
|
21
|
+
# Internal: Check if the 'private' directive is present.
|
22
|
+
def private?
|
23
|
+
@directives['private']
|
24
|
+
end
|
25
|
+
|
26
|
+
# Internal: Check if the 'no-cache' directive is present.
|
27
|
+
def no_cache?
|
28
|
+
@directives['no-cache']
|
29
|
+
end
|
30
|
+
|
31
|
+
# Internal: Check if the 'no-store' directive is present.
|
32
|
+
def no_store?
|
33
|
+
@directives['no-store']
|
34
|
+
end
|
35
|
+
|
36
|
+
# Internal: Get the 'max-age' directive as an Integer.
|
37
|
+
#
|
38
|
+
# Returns nil if the 'max-age' directive isn't present.
|
39
|
+
def max_age
|
40
|
+
@directives['max-age'].to_i if @directives.key?('max-age')
|
41
|
+
end
|
42
|
+
|
43
|
+
# Internal: Get the 'max-age' directive as an Integer.
|
44
|
+
#
|
45
|
+
# takes the age header integer value and reduces the max-age and s-maxage
|
46
|
+
# if present to account for having to remove static age header when caching responses
|
47
|
+
def normalize_max_ages(age)
|
48
|
+
if age > 0
|
49
|
+
@directives['max-age'] = @directives['max-age'].to_i - age if @directives.key?('max-age')
|
50
|
+
@directives['s-maxage'] = @directives['s-maxage'].to_i - age if @directives.key?('s-maxage')
|
51
|
+
end
|
52
|
+
end
|
53
|
+
|
54
|
+
# Internal: Get the 's-maxage' directive as an Integer.
|
55
|
+
#
|
56
|
+
# Returns nil if the 's-maxage' directive isn't present.
|
57
|
+
def shared_max_age
|
58
|
+
@directives['s-maxage'].to_i if @directives.key?('s-maxage')
|
59
|
+
end
|
60
|
+
alias_method :s_maxage, :shared_max_age
|
61
|
+
|
62
|
+
# Internal: Check if the 'must-revalidate' directive is present.
|
63
|
+
def must_revalidate?
|
64
|
+
@directives['must-revalidate']
|
65
|
+
end
|
66
|
+
|
67
|
+
# Internal: Check if the 'proxy-revalidate' directive is present.
|
68
|
+
def proxy_revalidate?
|
69
|
+
@directives['proxy-revalidate']
|
70
|
+
end
|
71
|
+
|
72
|
+
# Internal: Get the String representation for the cache directives.
|
73
|
+
# Directives are joined by a '=' and then combined into a single String
|
74
|
+
# separated by commas. Directives with a 'true' value will omit the '='
|
75
|
+
# sign and their value.
|
76
|
+
#
|
77
|
+
# Returns the Cache Control string.
|
78
|
+
def to_s
|
79
|
+
booleans, values = [], []
|
80
|
+
|
81
|
+
@directives.each do |key, value|
|
82
|
+
if value == true
|
83
|
+
booleans << key
|
84
|
+
elsif value
|
85
|
+
values << "#{key}=#{value}"
|
86
|
+
end
|
87
|
+
end
|
88
|
+
|
89
|
+
(booleans.sort + values.sort).join(', ')
|
90
|
+
end
|
91
|
+
|
92
|
+
private
|
93
|
+
|
94
|
+
# Internal: Parse the Cache Control string to a Hash.
|
95
|
+
# Existing whitespace will be removed and the string is split on commas.
|
96
|
+
# For each part everything before a '=' will be treated as the key
|
97
|
+
# and the exceeding will be treated as the value. If only the key is
|
98
|
+
# present then the assigned value will default to true.
|
99
|
+
#
|
100
|
+
# Examples:
|
101
|
+
# parse('max-age=600')
|
102
|
+
# # => { 'max-age' => '600'}
|
103
|
+
#
|
104
|
+
# parse('max-age')
|
105
|
+
# # => { 'max-age' => true }
|
106
|
+
#
|
107
|
+
# Returns a Hash.
|
108
|
+
def parse(header)
|
109
|
+
header.delete(' ').split(',').each_with_object({}) do |part, directives|
|
110
|
+
next if part.empty?
|
111
|
+
|
112
|
+
name, value = part.split('=', 2)
|
113
|
+
directives[name.downcase] = (value || true) unless name.empty?
|
114
|
+
end
|
115
|
+
end
|
116
|
+
end
|
117
|
+
end
|
118
|
+
end
|