dalli-elasticache 2.0.2 → 2.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +138 -0
- data/LICENSE +20 -0
- data/README.md +24 -7
- data/lib/dalli/elasticache/auto_discovery/base_command.rb +77 -13
- data/lib/dalli/elasticache/auto_discovery/config_response.rb +4 -3
- data/lib/dalli/elasticache/auto_discovery/endpoint.rb +17 -6
- data/lib/dalli/elasticache/auto_discovery/node.rb +2 -1
- data/lib/dalli/elasticache/version.rb +1 -1
- data/lib/dalli/elasticache.rb +20 -5
- metadata +7 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: ef06b82cfedd6c5a6faad37af6dff3bc6a5dba7205f9b074ab15dd5f53c89582
|
|
4
|
+
data.tar.gz: f0fd06b34b0c051658f34f6ad2c5d9ec7e377981493779be98cc17c3eca8a244
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 7c0fcc7900d628300a7fc406d3a586559d68fc29780e04bea3f9b69437d8e51682a08f181d5e7a102a7fb636b3a6359c5577c125d275be6383fa274e16027dce
|
|
7
|
+
data.tar.gz: 1a9244ff561d6e85479c61a4c17422c4d0a82df8e651f866a38f6c17ba6c20e21c33d1d520eb95055c36805ebb2f99352cf9c02dd5b83e219f180c9ae7cdd69c
|
data/CHANGELOG.md
ADDED
|
@@ -0,0 +1,138 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project will be documented in this file.
|
|
4
|
+
|
|
5
|
+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
|
6
|
+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
|
+
|
|
8
|
+
## [2.1.0]
|
|
9
|
+
|
|
10
|
+
### Changed
|
|
11
|
+
|
|
12
|
+
- `Endpoint` and the command classes now default to a 5 second timeout, matching `Dalli::ElastiCache`. Previously they had no timeout unless one was passed. Pass `timeout: nil` to disable it. [@petergoldstein](https://github.com/petergoldstein)
|
|
13
|
+
|
|
14
|
+
### Fixed
|
|
15
|
+
|
|
16
|
+
- Bracketed IPv6 configuration endpoints (e.g. `[::1]:11211`) were parsed but could never connect, because the brackets were passed through to address resolution. The brackets are now stripped before connecting, and SNI is no longer set when the host is an IP literal. [@petergoldstein](https://github.com/petergoldstein)
|
|
17
|
+
- An `ERROR`, `CLIENT_ERROR`, or `SERVER_ERROR` reply from the endpoint was not recognized, so discovery blocked until the read timeout (or indefinitely with `timeout: nil`). These replies now raise `Dalli::Elasticache::AutoDiscovery::ResponseError` immediately. The response terminator is now matched exactly instead of any line containing `END`. [@petergoldstein](https://github.com/petergoldstein)
|
|
18
|
+
- The TLS handshake with the configuration endpoint had no timeout, so an endpoint that accepted the TCP connection but never completed the handshake blocked discovery indefinitely, regardless of `timeout:`. The handshake is now bounded by `timeout:` and raises `Timeout::Error`. [@petergoldstein](https://github.com/petergoldstein)
|
|
19
|
+
- Node lists from IPv6 clusters, nodes reported without an IP address, and hostnames containing underscores were silently parsed as an empty list, producing a Dalli client with no servers that only failed on first use with `Dalli::RingError`. These nodes now parse correctly, and `#servers` (and therefore `#client`) raises `Dalli::Elasticache::AutoDiscovery::ResponseError` if a response contains no nodes. [@petergoldstein](https://github.com/petergoldstein)
|
|
20
|
+
- The configuration endpoint address was validated with line anchors, so a string containing a newline (e.g. `"junk\nhost:11211"`) parsed using one of its lines. Ports outside 1–65535 were also accepted. Both now raise `ArgumentError`. [@petergoldstein](https://github.com/petergoldstein)
|
|
21
|
+
- `Node#==` raised `NoMethodError` when compared with an object that is not a `Node`. It now returns `false`. [@petergoldstein](https://github.com/petergoldstein)
|
|
22
|
+
- Concurrent first calls to `Endpoint#config` or `Endpoint#stats` each ran their own network call. They now share one. [@petergoldstein](https://github.com/petergoldstein)
|
|
23
|
+
- `timeout:` now also bounds DNS resolution of the configuration endpoint (`resolv_timeout:`), not just the TCP connect. [@petergoldstein](https://github.com/petergoldstein)
|
|
24
|
+
- The README's "Refreshing the Node List" examples didn't work. `#refresh` doesn't update a `Dalli::Client` you already have, so the "on failure" example retried the same stale client indefinitely and the "on a schedule" example had no effect. The examples now build and switch to a new client, and the retry is limited. [@petergoldstein](https://github.com/petergoldstein)
|
|
25
|
+
- The published gem didn't include the `LICENSE` file (or `CHANGELOG.md`). Both are now packaged, and the gem metadata links to the source and changelog over https. [@petergoldstein](https://github.com/petergoldstein)
|
|
26
|
+
- `ResponseError` messages quoted the server's error line in full. Quoted text is now truncated to 200 characters. [@petergoldstein](https://github.com/petergoldstein)
|
|
27
|
+
|
|
28
|
+
### Security
|
|
29
|
+
|
|
30
|
+
- With `ssl_context:` set to `VERIFY_PEER`, the endpoint certificate was checked against the trusted CAs but not against the endpoint hostname, because `SSLContext.new` leaves `verify_hostname` false. Any certificate from a trusted CA was accepted for any host. The hostname (or IP address) is now verified whenever `verify_mode` includes `VERIFY_PEER`, even if `verify_hostname` is false, since an explicit `false` can't be distinguished from the default. Connections that relied on a certificate not matching the endpoint host will now fail. [@petergoldstein](https://github.com/petergoldstein)
|
|
31
|
+
- Discovery responses are now limited to 1 MB. Previously a misbehaving endpoint could send an unbounded response (or a single line with no terminator) and the client buffered all of it until the read timeout, or indefinitely with `timeout: nil`. Larger responses raise `Dalli::Elasticache::AutoDiscovery::ResponseError`. [@petergoldstein](https://github.com/petergoldstein)
|
|
32
|
+
|
|
33
|
+
## [2.0.2]
|
|
34
|
+
|
|
35
|
+
### Fixed
|
|
36
|
+
|
|
37
|
+
- Read timeout over TLS (`ssl_context:`) could block for the full timeout on every line after the first. `wait_readable` polled the raw TCP fd for readiness, which can't see data already buffered above it by OpenSSL, so it waited out the full timeout instead of returning immediately. Read timeouts are now enforced as a single deadline around the whole response rather than gating each line's readiness. [@petergoldstein](https://github.com/petergoldstein)
|
|
38
|
+
|
|
39
|
+
## [2.0.1]
|
|
40
|
+
|
|
41
|
+
### Fixed
|
|
42
|
+
|
|
43
|
+
- TLS support via `ssl_context:` keyword argument was advertised in v2.0.0 but was not included in the published gem due to a tagging error. [@petergoldstein](https://github.com/petergoldstein)
|
|
44
|
+
|
|
45
|
+
## [2.0.0]
|
|
46
|
+
|
|
47
|
+
### Added
|
|
48
|
+
|
|
49
|
+
- TCP connect and read timeout (default 5s) for auto-discovery calls. [@petergoldstein](https://github.com/petergoldstein)
|
|
50
|
+
- IPv6 endpoint support (bracketed addresses like `[::1]:11211`). [@petergoldstein](https://github.com/petergoldstein)
|
|
51
|
+
- TLS support for auto-discovery endpoint via `ssl_context:` keyword argument. [@petergoldstein](https://github.com/petergoldstein)
|
|
52
|
+
|
|
53
|
+
### Removed
|
|
54
|
+
|
|
55
|
+
- Legacy config command path for engines < 1.4.14. [@petergoldstein](https://github.com/petergoldstein)
|
|
56
|
+
- Support for Ruby < 3.3. [@petergoldstein](https://github.com/petergoldstein)
|
|
57
|
+
|
|
58
|
+
### Changed
|
|
59
|
+
|
|
60
|
+
- Fix `Dalli::Socket` shadowing `::Socket` in `BaseCommand`. [@petergoldstein](https://github.com/petergoldstein)
|
|
61
|
+
- Version requirements: Ruby v3.3+, and Dalli v5.0+. [@petergoldstein](https://github.com/petergoldstein)
|
|
62
|
+
|
|
63
|
+
## [1.0.1]
|
|
64
|
+
|
|
65
|
+
### Fixed
|
|
66
|
+
|
|
67
|
+
- Loading error because of incorrect path in a require_relative. [@petervandoros](https://github.com/petervandoros)
|
|
68
|
+
|
|
69
|
+
## [1.0.0]
|
|
70
|
+
|
|
71
|
+
### Added
|
|
72
|
+
|
|
73
|
+
- Rubocop linting. [@petergoldstein](https://github.com/petergoldstein)
|
|
74
|
+
- Specs for reading data off the socket, to ensure that code functions as expected. [@petergoldstein](https://github.com/petergoldstein)
|
|
75
|
+
- This CHANGELOG.md. [@petergoldstein](https://github.com/petergoldstein)
|
|
76
|
+
- Support for Google Cloud MemoryStore. [@petergoldstein](https://github.com/petergoldstein)
|
|
77
|
+
|
|
78
|
+
### Fixed
|
|
79
|
+
|
|
80
|
+
- Parsing error when retrieving the version. Version string is now parsed correctly. [@petergoldstein](https://github.com/petergoldstein)
|
|
81
|
+
- Library would error when the engine version was non-numeric or included additional data. Now treats such situations as a "modern" engine version. [@petergoldstein](https://github.com/petergoldstein)
|
|
82
|
+
|
|
83
|
+
### Changed
|
|
84
|
+
|
|
85
|
+
- BREAKING: engine_version is now returned as a string rather than a Gem::Version to support potentially non-numeric versions. [@petergoldstein](https://github.com/petergoldstein)
|
|
86
|
+
- Updated README to reflect deprecation of DalliStore and preferred use of MemCacheStore. [@xiaoronglv](https://github.com/xiaoronglv)
|
|
87
|
+
- Switched to GitHub Actions from Travis for CI. Added Ruby 2.6, 2.7, 3.0, 3.1, ruby-head, jruby-9.3, jruby-head. [@petergoldstein](https://github.com/petergoldstein)
|
|
88
|
+
- Dalli::Elasticache now raises an ArgumentError if it cannot parse the config endpoint argument. [@petergoldstein](https://github.com/petergoldstein)
|
|
89
|
+
- Now use default port of 11211 for configuration endpoint when not explicitly specified. [@petergoldstein](https://github.com/petergoldstein)
|
|
90
|
+
- Refactored internal classes to better enable testing, shrink individual class responsibilities. [@petergoldstein](https://github.com/petergoldstein)
|
|
91
|
+
- Allow underscores in hostnames.
|
|
92
|
+
|
|
93
|
+
### Removed
|
|
94
|
+
|
|
95
|
+
- Support for all Rubies before 2.6 was dropped. [@petergoldstein](https://github.com/petergoldstein)
|
|
96
|
+
|
|
97
|
+
|
|
98
|
+
## [0.2.0] - 2016-02-24
|
|
99
|
+
|
|
100
|
+
### Changed
|
|
101
|
+
|
|
102
|
+
- Node connections now use hostnames as opposed to IPs (which may change over time). [@BanjoInc](https://github.com/BanjoInc)
|
|
103
|
+
- Ruby 2.2 and 2.3 was added to CI. [@ktheory](https://github.com/ktheory)
|
|
104
|
+
|
|
105
|
+
### Removed
|
|
106
|
+
|
|
107
|
+
- Support for Ruby 1.9.2 and 1.9.3 was dropped. [@ktheory](https://github.com/ktheory)
|
|
108
|
+
|
|
109
|
+
## [0.1.2] - 2014-07-08
|
|
110
|
+
|
|
111
|
+
### Changed
|
|
112
|
+
|
|
113
|
+
- Added Ruby 2.0 and 2.1 to CI. [@petergoldstein](https://github.com/petergoldstein)
|
|
114
|
+
|
|
115
|
+
### Fixed
|
|
116
|
+
|
|
117
|
+
- Addressed NameError on refresh. [@ryo0301](https://github.com/ryo0301)
|
|
118
|
+
|
|
119
|
+
## [0.1.1] - 2014-05-03
|
|
120
|
+
|
|
121
|
+
### Added
|
|
122
|
+
|
|
123
|
+
- Ability to retrieve configuration version (indication of how many times the node set has changed) from endpoint. [@zmillman](https://github.com/zmillman)
|
|
124
|
+
- Specs for existing functionality. [@zmillman](https://github.com/zmillman)
|
|
125
|
+
- Continuous Integration using Travis CI. [@zmillman](https://github.com/zmillman)
|
|
126
|
+
- Refresh capability for the node set. [@zmillman](https://github.com/zmillman)
|
|
127
|
+
|
|
128
|
+
### Changed
|
|
129
|
+
|
|
130
|
+
- Refactoring and repackaging of the endpoint classes. [@zmillman](https://github.com/zmillman)
|
|
131
|
+
|
|
132
|
+
|
|
133
|
+
## [0.1.0] - 2013-01-27
|
|
134
|
+
|
|
135
|
+
### Added
|
|
136
|
+
|
|
137
|
+
- Initial implementation for fetching node addresses from an Amazon ElastiCache endpoint. [@ktheory](https://github.com/ktheory)
|
|
138
|
+
|
data/LICENSE
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
Copyright (2017-2022) Aaron Suggs, Peter M. Goldstein
|
|
2
|
+
|
|
3
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
|
4
|
+
a copy of this software and associated documentation files (the
|
|
5
|
+
"Software"), to deal in the Software without restriction, including
|
|
6
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
|
7
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
|
8
|
+
permit persons to whom the Software is furnished to do so, subject to
|
|
9
|
+
the following conditions:
|
|
10
|
+
|
|
11
|
+
The above copyright notice and this permission notice shall be
|
|
12
|
+
included in all copies or substantial portions of the Software.
|
|
13
|
+
|
|
14
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
|
15
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
|
16
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
|
17
|
+
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
|
18
|
+
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
|
19
|
+
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
|
20
|
+
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README.md
CHANGED
|
@@ -16,7 +16,7 @@ gem 'dalli-elasticache'
|
|
|
16
16
|
Using Dalli Elasticache in Rails
|
|
17
17
|
---------------------------------
|
|
18
18
|
|
|
19
|
-
Note that the list of memcached servers used by Rails will be refreshed each time an app server process starts. If the list of nodes in your cluster changes, this configuration will not be reflected in the Rails
|
|
19
|
+
Note that the list of memcached servers used by Rails will be refreshed each time an app server process starts. If the list of nodes in your cluster changes, this configuration will not be reflected in the Rails configuration without such a server process restart.
|
|
20
20
|
|
|
21
21
|
### Configuring a Cache Store
|
|
22
22
|
|
|
@@ -95,7 +95,7 @@ Pass an `OpenSSL::SSL::SSLContext` via `ssl_context:` to enable TLS for the auto
|
|
|
95
95
|
|
|
96
96
|
```ruby
|
|
97
97
|
ssl_context = OpenSSL::SSL::SSLContext.new
|
|
98
|
-
ssl_context.verify_mode
|
|
98
|
+
ssl_context.set_params(verify_mode: OpenSSL::SSL::VERIFY_PEER)
|
|
99
99
|
|
|
100
100
|
elasticache = Dalli::ElastiCache.new(
|
|
101
101
|
"my-cluster.abc123.cfg.use1.cache.amazonaws.com:11211",
|
|
@@ -104,6 +104,8 @@ elasticache = Dalli::ElastiCache.new(
|
|
|
104
104
|
)
|
|
105
105
|
```
|
|
106
106
|
|
|
107
|
+
When the context's `verify_mode` includes `VERIFY_PEER`, the endpoint's certificate is also checked against the endpoint hostname (or IP address), even if `verify_hostname` was left at its `SSLContext.new` default of `false`. `set_params` additionally loads the system CA store.
|
|
108
|
+
|
|
107
109
|
Note that `ssl_context:` controls TLS for the auto-discovery endpoint only. To also encrypt traffic to the cache nodes themselves, pass `ssl_context:` in the Dalli options as well:
|
|
108
110
|
|
|
109
111
|
```ruby
|
|
@@ -112,32 +114,47 @@ elasticache = Dalli::ElastiCache.new(endpoint, { ssl_context: ssl_context }, ssl
|
|
|
112
114
|
|
|
113
115
|
### Refreshing the Node List
|
|
114
116
|
|
|
115
|
-
The node list is fetched once and cached.
|
|
117
|
+
The node list is fetched once and cached. `#refresh` discards the cached list so the next call re-queries the endpoint. It does **not** update a `Dalli::Client` you already have: each call to `#client` builds a new client from the node list at that moment. After refreshing, call `#client` again and switch to the new client. You can then `close` the old one; Dalli reconnects a closed client automatically if anything still uses it.
|
|
116
118
|
|
|
117
119
|
Two common patterns for keeping the node list up to date:
|
|
118
120
|
|
|
119
|
-
**On failure
|
|
121
|
+
**On failure:** rescue `Dalli::RingError` (raised when no nodes are reachable), build a new client from a refreshed node list, and retry once:
|
|
120
122
|
|
|
121
123
|
```ruby
|
|
124
|
+
retried = false
|
|
122
125
|
begin
|
|
123
126
|
cache.get("key")
|
|
124
127
|
rescue Dalli::RingError
|
|
125
|
-
|
|
128
|
+
raise if retried
|
|
129
|
+
|
|
130
|
+
retried = true
|
|
131
|
+
old_cache = cache
|
|
132
|
+
cache = elasticache.refresh.client
|
|
133
|
+
old_cache.close
|
|
126
134
|
retry
|
|
127
135
|
end
|
|
128
136
|
```
|
|
129
137
|
|
|
130
|
-
|
|
138
|
+
Limit the retries. If the cluster is still unreachable after a refresh, retrying without a limit loops forever.
|
|
139
|
+
|
|
140
|
+
**On a schedule:** refresh periodically in a background thread or job and swap in the new client. AWS does not publish exact timing for node replacement, but the process typically takes a few minutes, so refreshing every 60 seconds is a reasonable default:
|
|
131
141
|
|
|
132
142
|
```ruby
|
|
133
143
|
Thread.new do
|
|
134
144
|
loop do
|
|
135
145
|
sleep 60
|
|
136
|
-
elasticache.refresh
|
|
146
|
+
new_cache = elasticache.refresh.client
|
|
147
|
+
old_cache = cache
|
|
148
|
+
cache = new_cache
|
|
149
|
+
old_cache.close
|
|
150
|
+
rescue StandardError => e
|
|
151
|
+
warn "ElastiCache refresh failed, keeping the current client: #{e.message}"
|
|
137
152
|
end
|
|
138
153
|
end
|
|
139
154
|
```
|
|
140
155
|
|
|
156
|
+
If discovery fails, for example because the endpoint times out or lists no nodes, `#client` raises and the existing client stays in use.
|
|
157
|
+
|
|
141
158
|
A combination of both approaches is reasonable for production use.
|
|
142
159
|
|
|
143
160
|
License
|
|
@@ -1,11 +1,19 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require 'ipaddr'
|
|
3
4
|
require 'openssl'
|
|
4
5
|
require 'timeout'
|
|
5
6
|
|
|
6
7
|
module Dalli
|
|
7
8
|
module Elasticache
|
|
8
9
|
module AutoDiscovery
|
|
10
|
+
##
|
|
11
|
+
# Raised when the configuration endpoint's response can't be used: it replied
|
|
12
|
+
# with ERROR, CLIENT_ERROR, or SERVER_ERROR, the response exceeded
|
|
13
|
+
# MAX_RESPONSE_BYTES, or (from Dalli::ElastiCache#servers) it listed no nodes.
|
|
14
|
+
##
|
|
15
|
+
class ResponseError < StandardError; end
|
|
16
|
+
|
|
9
17
|
##
|
|
10
18
|
# Base command class for configuration endpoint
|
|
11
19
|
# command. Contains the network logic.
|
|
@@ -13,7 +21,18 @@ module Dalli
|
|
|
13
21
|
class BaseCommand
|
|
14
22
|
attr_reader :host, :port, :timeout, :ssl_context
|
|
15
23
|
|
|
16
|
-
|
|
24
|
+
DEFAULT_TIMEOUT = 5
|
|
25
|
+
|
|
26
|
+
# Upper bound on a response. Even a large cluster's node list is a few tens of KB.
|
|
27
|
+
MAX_RESPONSE_BYTES = 1024 * 1024
|
|
28
|
+
|
|
29
|
+
# Server-supplied text quoted in an exception message is cut to this length
|
|
30
|
+
MAX_ERROR_TEXT_LENGTH = 200
|
|
31
|
+
|
|
32
|
+
END_LINE = "END\r\n"
|
|
33
|
+
ERROR_LINE_REGEX = /\A(?:ERROR|CLIENT_ERROR|SERVER_ERROR)\b/
|
|
34
|
+
|
|
35
|
+
def initialize(host, port, timeout = DEFAULT_TIMEOUT, ssl_context: nil)
|
|
17
36
|
@host = host
|
|
18
37
|
@port = port
|
|
19
38
|
@timeout = timeout
|
|
@@ -24,7 +43,7 @@ module Dalli
|
|
|
24
43
|
#
|
|
25
44
|
# Returns the raw response as a String
|
|
26
45
|
def send_command
|
|
27
|
-
tcp_socket = ::Socket.tcp(
|
|
46
|
+
tcp_socket = ::Socket.tcp(socket_host, @port, connect_timeout: timeout, resolv_timeout: timeout)
|
|
28
47
|
begin
|
|
29
48
|
socket = ssl_context ? wrap_with_ssl(tcp_socket) : tcp_socket
|
|
30
49
|
socket.puts command
|
|
@@ -35,38 +54,83 @@ module Dalli
|
|
|
35
54
|
end
|
|
36
55
|
|
|
37
56
|
# `timeout` bounds the entire multi-line read as a single wall-clock deadline,
|
|
38
|
-
# not each individual
|
|
57
|
+
# not each individual line read. A per-line readiness check (e.g. `wait_readable`)
|
|
39
58
|
# can't tell buffered-but-unread bytes apart from an empty socket once a buffering
|
|
40
59
|
# layer (Ruby's IO buffer, or OpenSSL::Buffering for TLS) sits between the fd and
|
|
41
|
-
#
|
|
60
|
+
# the line read, so it can't be used to gate individual lines reliably.
|
|
42
61
|
def response_from_socket(socket)
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
Timeout.timeout(timeout, Timeout::Error, "Auto discovery read timed out after #{timeout}s") do
|
|
46
|
-
read_lines(socket)
|
|
47
|
-
end
|
|
62
|
+
with_timeout('read') { read_lines(socket) }
|
|
48
63
|
end
|
|
49
64
|
|
|
50
65
|
private
|
|
51
66
|
|
|
67
|
+
def with_timeout(phase, &)
|
|
68
|
+
return yield unless timeout
|
|
69
|
+
|
|
70
|
+
Timeout.timeout(timeout, Timeout::Error, "Auto discovery #{phase} timed out after #{timeout}s", &)
|
|
71
|
+
end
|
|
72
|
+
|
|
73
|
+
# The host without the brackets used to delimit an IPv6 address from its port
|
|
74
|
+
def socket_host
|
|
75
|
+
@host.delete_prefix('[').delete_suffix(']')
|
|
76
|
+
end
|
|
77
|
+
|
|
78
|
+
def ip_address?(host)
|
|
79
|
+
IPAddr.new(host)
|
|
80
|
+
true
|
|
81
|
+
rescue IPAddr::InvalidAddressError
|
|
82
|
+
false
|
|
83
|
+
end
|
|
84
|
+
|
|
52
85
|
def read_lines(socket)
|
|
53
86
|
data = +''
|
|
54
87
|
loop do
|
|
55
|
-
line = socket.
|
|
56
|
-
break if line
|
|
88
|
+
line = next_line(socket, data.bytesize)
|
|
89
|
+
break if line == END_LINE
|
|
90
|
+
raise ResponseError, "Auto discovery command failed: #{error_text(line)}" if ERROR_LINE_REGEX.match?(line)
|
|
57
91
|
|
|
58
92
|
data << line
|
|
59
93
|
end
|
|
60
94
|
data
|
|
61
95
|
end
|
|
62
96
|
|
|
97
|
+
def error_text(line)
|
|
98
|
+
text = line.strip
|
|
99
|
+
text.length > MAX_ERROR_TEXT_LENGTH ? "#{text[0, MAX_ERROR_TEXT_LENGTH]}..." : text
|
|
100
|
+
end
|
|
101
|
+
|
|
102
|
+
# Reads one line, or as much of one as fits in the remaining response budget.
|
|
103
|
+
# The limit is one byte over the budget so an oversized response is detectable.
|
|
104
|
+
# `gets` is used because OpenSSL's `readline` doesn't accept a limit.
|
|
105
|
+
def next_line(socket, bytes_read)
|
|
106
|
+
line = socket.gets("\n", MAX_RESPONSE_BYTES - bytes_read + 1)
|
|
107
|
+
raise EOFError, 'Auto discovery connection closed before END' unless line
|
|
108
|
+
if bytes_read + line.bytesize > MAX_RESPONSE_BYTES
|
|
109
|
+
raise ResponseError, "Auto discovery response exceeded #{MAX_RESPONSE_BYTES} bytes"
|
|
110
|
+
end
|
|
111
|
+
|
|
112
|
+
line
|
|
113
|
+
end
|
|
114
|
+
|
|
63
115
|
def wrap_with_ssl(tcp_socket)
|
|
64
116
|
ssl_socket = OpenSSL::SSL::SSLSocket.new(tcp_socket, ssl_context)
|
|
65
|
-
|
|
117
|
+
# SNI must be a DNS name (RFC 6066), never an IP literal
|
|
118
|
+
ssl_socket.hostname = socket_host unless ip_address?(socket_host)
|
|
66
119
|
ssl_socket.sync_close = true
|
|
67
|
-
|
|
120
|
+
# connect_timeout only covers the TCP connect, so bound the handshake separately
|
|
121
|
+
with_timeout('TLS handshake') { ssl_socket.connect }
|
|
122
|
+
verify_hostname(ssl_socket)
|
|
68
123
|
ssl_socket
|
|
69
124
|
end
|
|
125
|
+
|
|
126
|
+
# SSLContext.new leaves verify_hostname false, so VERIFY_PEER alone only checks
|
|
127
|
+
# that the certificate chains to a trusted CA, not that it was issued for this
|
|
128
|
+
# host. Check the hostname explicitly whenever the peer is being verified.
|
|
129
|
+
def verify_hostname(ssl_socket)
|
|
130
|
+
return if ssl_context.verify_mode.to_i.nobits?(OpenSSL::SSL::VERIFY_PEER)
|
|
131
|
+
|
|
132
|
+
ssl_socket.post_connection_check(socket_host)
|
|
133
|
+
end
|
|
70
134
|
end
|
|
71
135
|
end
|
|
72
136
|
end
|
|
@@ -14,8 +14,9 @@ module Dalli
|
|
|
14
14
|
# Matches the version line of the response
|
|
15
15
|
VERSION_REGEX = /^(\d+)\r?\n/
|
|
16
16
|
|
|
17
|
-
# Matches strings like "my-cluster.001.cache.aws.com|10.154.182.29|11211"
|
|
18
|
-
|
|
17
|
+
# Matches strings like "my-cluster.001.cache.aws.com|10.154.182.29|11211".
|
|
18
|
+
# The IP may be IPv4, IPv6, or empty; only the host and port are used to connect.
|
|
19
|
+
NODE_REGEX = /(([-.\w]+)\|([^|\s]*)\|(\d+))/
|
|
19
20
|
NODE_LIST_REGEX = /^(#{NODE_REGEX}\s*)+$/
|
|
20
21
|
|
|
21
22
|
def initialize(response_text)
|
|
@@ -34,7 +35,7 @@ module Dalli
|
|
|
34
35
|
|
|
35
36
|
# Node hosts, ip addresses, and ports
|
|
36
37
|
#
|
|
37
|
-
# Returns an Array of
|
|
38
|
+
# Returns an Array of Node
|
|
38
39
|
def nodes
|
|
39
40
|
NODE_LIST_REGEX.match(@text).to_s.scan(NODE_REGEX).map do |match|
|
|
40
41
|
Node.new(match[1], match[2], match[3].to_i)
|
|
@@ -13,12 +13,13 @@ module Dalli
|
|
|
13
13
|
attr_reader :host, :port
|
|
14
14
|
|
|
15
15
|
# Matches DNS/IPv4 like "my-host.cache.aws.com:11211" or bracketed IPv6 like "[::1]:11211"
|
|
16
|
-
ENDPOINT_REGEX =
|
|
16
|
+
ENDPOINT_REGEX = /\A([-_.a-zA-Z0-9]+|\[[a-fA-F0-9:]+\])(?::(\d+))?\z/
|
|
17
17
|
|
|
18
|
-
def initialize(addr, timeout:
|
|
18
|
+
def initialize(addr, timeout: BaseCommand::DEFAULT_TIMEOUT, ssl_context: nil)
|
|
19
19
|
@host, @port = parse_endpoint_address(addr)
|
|
20
20
|
@timeout = timeout
|
|
21
21
|
@ssl_context = ssl_context
|
|
22
|
+
@mutex = Mutex.new
|
|
22
23
|
end
|
|
23
24
|
|
|
24
25
|
DEFAULT_PORT = 11_211
|
|
@@ -26,17 +27,27 @@ module Dalli
|
|
|
26
27
|
m = ENDPOINT_REGEX.match(addr)
|
|
27
28
|
raise ArgumentError, "Unable to parse configuration endpoint address - #{addr}" unless m
|
|
28
29
|
|
|
29
|
-
|
|
30
|
+
port = (m[2] || DEFAULT_PORT).to_i
|
|
31
|
+
unless (1..65_535).cover?(port)
|
|
32
|
+
raise ArgumentError, "Invalid configuration endpoint address - #{addr}: port must be between 1 and 65535"
|
|
33
|
+
end
|
|
34
|
+
|
|
35
|
+
[m[1], port]
|
|
30
36
|
end
|
|
31
37
|
|
|
32
|
-
# A cached ElastiCache::StatsResponse
|
|
38
|
+
# A cached ElastiCache::StatsResponse. The mutex ensures concurrent first
|
|
39
|
+
# callers share one network call instead of each running their own.
|
|
33
40
|
def stats
|
|
34
|
-
@stats
|
|
41
|
+
@stats || @mutex.synchronize do
|
|
42
|
+
@stats ||= StatsCommand.new(@host, @port, @timeout, ssl_context: @ssl_context).response
|
|
43
|
+
end
|
|
35
44
|
end
|
|
36
45
|
|
|
37
46
|
# A cached ElastiCache::ConfigResponse
|
|
38
47
|
def config
|
|
39
|
-
@config
|
|
48
|
+
@config || @mutex.synchronize do
|
|
49
|
+
@config ||= ConfigCommand.new(@host, @port, @timeout, ssl_context: @ssl_context).response
|
|
50
|
+
end
|
|
40
51
|
end
|
|
41
52
|
|
|
42
53
|
# The memcached engine version
|
data/lib/dalli/elasticache.rb
CHANGED
|
@@ -22,7 +22,7 @@ module Dalli
|
|
|
22
22
|
# distributes cached items consistently over the nodes.
|
|
23
23
|
##
|
|
24
24
|
class ElastiCache
|
|
25
|
-
DEFAULT_TIMEOUT =
|
|
25
|
+
DEFAULT_TIMEOUT = Dalli::Elasticache::AutoDiscovery::BaseCommand::DEFAULT_TIMEOUT
|
|
26
26
|
|
|
27
27
|
attr_reader :endpoint, :options
|
|
28
28
|
|
|
@@ -35,8 +35,10 @@ module Dalli
|
|
|
35
35
|
# a bracketed IPv6 address.
|
|
36
36
|
# dalli_options - a set of options passed to the Dalli::Client that is returned
|
|
37
37
|
# by the client method. Otherwise unused.
|
|
38
|
-
# timeout: -
|
|
39
|
-
#
|
|
38
|
+
# timeout: - timeout in seconds applied separately to each step of an auto-discovery
|
|
39
|
+
# call: DNS resolution, TCP connect, TLS handshake, and reading the response.
|
|
40
|
+
# Defaults to 5 seconds. Pass nil to disable.
|
|
41
|
+
# ssl_context: - an OpenSSL::SSL::SSLContext to use TLS for the auto-discovery calls.
|
|
40
42
|
##
|
|
41
43
|
def initialize(config_endpoint, dalli_options = {}, timeout: DEFAULT_TIMEOUT, ssl_context: nil)
|
|
42
44
|
@timeout = timeout
|
|
@@ -66,11 +68,24 @@ module Dalli
|
|
|
66
68
|
|
|
67
69
|
# List of cluster server nodes with ip addresses and ports
|
|
68
70
|
# Always use host name instead of private elasticache IPs as internal IPs can change after a node is rebooted
|
|
71
|
+
#
|
|
72
|
+
# Raises Dalli::Elasticache::AutoDiscovery::ResponseError if the response contains
|
|
73
|
+
# no nodes, rather than handing Dalli an empty server list that only fails on first use
|
|
69
74
|
def servers
|
|
70
|
-
endpoint.config.nodes
|
|
75
|
+
nodes = endpoint.config.nodes
|
|
76
|
+
if nodes.empty?
|
|
77
|
+
raise Dalli::Elasticache::AutoDiscovery::ResponseError,
|
|
78
|
+
"No cache nodes found in auto discovery response from #{endpoint.host}:#{endpoint.port}"
|
|
79
|
+
end
|
|
80
|
+
|
|
81
|
+
nodes.map(&:to_s)
|
|
71
82
|
end
|
|
72
83
|
|
|
73
|
-
#
|
|
84
|
+
# Discards the cached discovery results so the next call re-queries the
|
|
85
|
+
# configuration endpoint. Returns self.
|
|
86
|
+
#
|
|
87
|
+
# This does not change Dalli::Client instances already returned by #client;
|
|
88
|
+
# call #client again to get one that uses the current node list.
|
|
74
89
|
def refresh
|
|
75
90
|
config_endpoint = "#{endpoint.host}:#{endpoint.port}"
|
|
76
91
|
@endpoint = Dalli::Elasticache::AutoDiscovery::Endpoint.new(config_endpoint, timeout: @timeout,
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: dalli-elasticache
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 2.0
|
|
4
|
+
version: 2.1.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Aaron Suggs
|
|
@@ -37,6 +37,8 @@ executables: []
|
|
|
37
37
|
extensions: []
|
|
38
38
|
extra_rdoc_files: []
|
|
39
39
|
files:
|
|
40
|
+
- CHANGELOG.md
|
|
41
|
+
- LICENSE
|
|
40
42
|
- README.md
|
|
41
43
|
- Rakefile
|
|
42
44
|
- lib/dalli-elasticache.rb
|
|
@@ -49,11 +51,13 @@ files:
|
|
|
49
51
|
- lib/dalli/elasticache/auto_discovery/stats_command.rb
|
|
50
52
|
- lib/dalli/elasticache/auto_discovery/stats_response.rb
|
|
51
53
|
- lib/dalli/elasticache/version.rb
|
|
52
|
-
homepage:
|
|
54
|
+
homepage: https://github.com/ktheory/dalli-elasticache
|
|
53
55
|
licenses:
|
|
54
56
|
- MIT
|
|
55
57
|
metadata:
|
|
56
58
|
rubygems_mfa_required: 'true'
|
|
59
|
+
source_code_uri: https://github.com/ktheory/dalli-elasticache
|
|
60
|
+
changelog_uri: https://github.com/ktheory/dalli-elasticache/blob/master/CHANGELOG.md
|
|
57
61
|
rdoc_options:
|
|
58
62
|
- "--charset=UTF-8"
|
|
59
63
|
require_paths:
|
|
@@ -69,7 +73,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
69
73
|
- !ruby/object:Gem::Version
|
|
70
74
|
version: 1.3.5
|
|
71
75
|
requirements: []
|
|
72
|
-
rubygems_version: 4.0.
|
|
76
|
+
rubygems_version: 4.0.23
|
|
73
77
|
specification_version: 4
|
|
74
78
|
summary: Configure Dalli clients with ElastiCache's AutoDiscovery
|
|
75
79
|
test_files: []
|