dalli 3.2.13 → 4.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 7f7bcbe37e631c3c733d1487e519501497702b873e81e6588d6dfad1ab96a65b
4
- data.tar.gz: d7a953d2c0052334d8edb7fa45306724ffa25cec64436dcceccc9847d057e8bd
3
+ metadata.gz: 86e39b2869b7c916472e88e6cbbfb6aa1f6533aa35c9e1f1c704cffcbc680f8a
4
+ data.tar.gz: bf2e56610c6bae187d561ccf09105e6f5b4a29eed1308d089745ef22cf5b673b
5
5
  SHA512:
6
- metadata.gz: 8d7b0ed94102d333f611736c48d944b0f1010760ae646003db60cd02c90b1a047a5fd10f7a5ef077527a88594f8844cbe08f9647259749e78ade74d09d2e851b
7
- data.tar.gz: bf86e4604fc18eaf2bdf57d6abad82258745238112a9866e5da9e05c0f45b44d62f82ebe8b5519ecc83d00ec6ee94e2d11bb931aa14171081a0b731745633e6d
6
+ metadata.gz: e2faab814abb4b25a53d87048f7e4e81e49c609bd64725cb18607670110da3d5b8038544598c44575831b9038500f9d8220a56ec28c16b9517a21d442212d12b
7
+ data.tar.gz: 3e5a7c326c908d72d2d8ce2211b27f67acebbbf97c279c5faf3ba2914ec7409df7f259a9082d82903cc9ae6b6716a137994aba849321546b8d3d3a35f8f7f5af
data/CHANGELOG.md CHANGED
@@ -1,82 +1,30 @@
1
1
  Dalli Changelog
2
2
  =====================
3
3
 
4
- Unreleased
5
- ==========
6
-
7
- 3.2.13
8
- ==========
9
-
10
- - Complete the fix for per-request `raw: true` on reads (GHSA-wr87-m4jw-29x5)
11
- - `cas` and `cas!` accepted `raw: true` but didn't pass it to the read, so the value was still deserialized. They now honor it, in both protocols
12
- - Affects 3.2.12 and earlier; fixed in 5.2.2, 5.1.4, 5.0.10, 4.3.7 and 3.2.13
13
- - Complete the fork-safety fix (GHSA-w39f-xq2m-4g8x)
14
- - Fork detection used a pid cache refreshed by a `Process._fork` hook. When another library's fork hook ran first in the child (connection_pool, loaded before Dalli as in Rails, closes its connections there), Dalli didn't know it was in a forked child and ended the parent's TLS session. It now checks `Process.pid`
15
- - Fixed in 4.3.7 and 3.2.13
16
-
17
- - Fix a regression in 3.2.12: a `get_multi` that didn't finish within `socket_timeout` counted toward `socket_max_failures`, so two slow `get_multi` calls in a row marked a healthy server down. It now just closes the connection
18
- - Handle malformed pipelined replies from a broken or hostile server or proxy: a hit with no key, or with no `s` flag, could leave the connection out of step or raise an error
19
- - With `decompressed_max_bytes` in effect, data after the end of a compressed value's stream was returned as part of the value. It's now ignored, as without the limit
20
- - With a `digest_class` whose digests aren't short hex strings, shortening a long key could loop forever. It now raises `ArgumentError`
21
-
22
- - Run the Tests and RuboCop workflows on pushes to `main` and the `*-stable` branches only, so a pull request's branch isn't tested twice (backport of #1198)
23
-
24
- 3.2.12
25
- ==========
26
-
27
- - Fix pipelined `get_multi` returning one key's value for another after an error reply (GHSA-p6pm-ch9v-44vx)
28
- - The pipelined reply parser took any reply without a body for the end of the batch. An error reply for one key (`CLIENT_ERROR`, `SERVER_ERROR`, or `EN` from a proxy) ended it early, and the replies still on the connection were read as the replies to later commands, so a `get` could return another key's value
29
- - Keys are now measured in bytes as sent (after base64 encoding when the meta protocol needs it). A key under 250 characters but over 250 bytes, for example one with many non-ASCII characters, passed validation:
30
- - with the meta protocol, memcached rejected it with the error reply above
31
- - with the binary protocol, memcached dropped the connection, and `get` or `get_multi` with that key retried forever
32
- - Such keys are now truncated like other long keys; keys that worked before are unchanged
33
- - The reply parser affects `protocol: :meta` since 3.2.0; the key length affects both protocols; fixed in 5.2.1, 5.1.3, 5.0.9, 4.3.6 and 3.2.12
34
- - Stop retrying an unresponsive server forever (GHSA-4qp6-2jcr-596v)
35
- - A server that accepted connections but never answered (or dropped the connection on a request) was retried forever, hanging the caller, because each successful reconnect reset the failure count. Requests now fail after `socket_max_failures` attempts and the server is marked down, as when it can't be reached at all; it gets a full set of attempts again after `down_retry_delay`
36
- - Affects all versions; fixed in 5.2.1, 5.1.3, 5.0.9, 4.3.6 and 3.2.12
37
- - Limit decompressed and reply value sizes (GHSA-3553-vcg5-72jw)
38
- - Values flagged as compressed were inflated without limit, so a small stored value could expand to gigabytes on read, whatever the client's `compress` or `serializer` settings. The new `decompressed_max_bytes` option (default 128 MiB; `nil` disables it) makes a read that would pass it raise `Dalli::UnmarshalError`. Custom compressors whose `decompress` takes only the data keep working, without the limit
39
- - The value size in a reply (a meta `VA` size or a binary body length) was used to read or buffer that many bytes, so a malicious server could make the client allocate gigabytes. Sizes over 1 GiB, memcached's largest item, now raise `Dalli::DalliError` before reading
40
- - Affects all versions; fixed in 5.2.1, 5.1.3, 5.0.9, 4.3.6 and 3.2.12
41
- - Honor per-request `raw: true` on reads, and add `Dalli::JSONSerializer` (GHSA-wr87-m4jw-29x5)
42
- - `get`, `gat` and `fetch` ignored a per-request `raw: true` and deserialized the value according to its stored flags, so a caller who asked for raw bytes could still have `Marshal.load` run on data someone else wrote to memcached. They now return the stored bytes
43
- - The `raw` part affects all versions
44
- - `serializer: JSON` reads values with `JSON.load`, which on json gem versions before 3.0 creates an object of the class named in a stored `json_class` key when the json additions are loaded. The new `Dalli::JSONSerializer` reads with `JSON.parse` and only returns plain JSON types. The README now recommends it; `serializer: JSON` itself is unchanged
45
- - The `serializer: JSON` part affects any version used with the json gem before 3.0
46
- - Keep a forked child from closing the parent's TLS session (GHSA-w39f-xq2m-4g8x)
47
- - With TLS, a forked child that closed the client (or reconnected after detecting the fork) sent a TLS close on the connection the parent was still using, ending the parent's session. A forked child now closes only its own copy of the socket
48
- - Affects all versions with TLS; fixed in 5.2.1, 5.1.3, 5.0.9, 4.3.6 and 3.2.12
49
-
50
- - Fix three flaky tests: the `KeyManager` namespace key-length test, the Rack session freshness test, and `MemcachedManager`'s start and stop handling (backport of #1141, #1183)
51
- - Fix flaky failover tests: move their ports out of Linux's ephemeral port range, and wait for memcached to accept connections after starting it (backport of #1184)
52
-
53
- 3.2.11
54
- ==========
55
-
56
- - Fix multi-server `get_multi` stopping at an empty value (#1170)
57
- - The pipelined reply parser took a hit on a zero-length value (`VA 0`) for the terminating `MN`, so that server's remaining keys were silently missing from the result
58
- - Its value terminator and any replies not yet parsed stayed on the connection and were read as the replies to later commands on it. Those commands could fail with `Dalli::DalliError: Response error`, or a single-key `get` could silently return another key's value
59
- - An empty value is now returned as `''`, like a single-key `get` and single-server `get_multi`
60
- - Affects only clients using `protocol: :meta`; the default binary protocol is not affected
61
- - Thanks to Julian Richard Contreras for this contribution
62
-
63
- 3.2.10
64
- ==========
65
-
66
- - Fix `Rack::Session::Dalli` with connection_pool 3.x, which only accepts keyword arguments (backport of #1051)
67
- - Add `logger` as a runtime dependency, since it is no longer a default gem as of Ruby 4.0 (backport of #1010, olleolleolle)
68
- - CI: patch memcached 1.5.22 so it builds on ubuntu-22.04, and run tests there (backport of #1033)
69
-
70
- 3.2.9
71
- ==========
72
-
73
- Security:
74
-
75
- - Fix memcached command injection through the `incr`/`decr` default with the meta protocol (GHSA-6wmv-xq9m-fmp7)
76
- - With `protocol: :meta`, the `default` argument of `incr`/`decr` was written into the command without conversion, so a String containing CRLF injected additional memcached commands (e.g. `set`, `flush_all`) on the connection. The default binary protocol is not affected
77
- - `default` must now be an Integer, or a String of decimal digits; anything else raises `ArgumentError` before a request is sent
78
- - As defense in depth, the meta `RequestFormatter` now converts every numeric flag it writes (`D`, `J`, `N`, `T`) to an Integer
79
- - Thanks to oss-security-shop for the report
4
+ 4.0.0
5
+ ==========
6
+
7
+ BREAKING CHANGES:
8
+
9
+ - Require Ruby 3.1+ (dropped support for Ruby 2.6, 2.7, and 3.0)
10
+ - Removed `Dalli::Server` deprecated alias - use `Dalli::Protocol::Binary` instead
11
+ - Removed `:compression` option - use `:compress` instead
12
+ - Removed `close_on_fork` method - use `reconnect_on_fork` instead
13
+
14
+ Other changes:
15
+
16
+ - Add security warning when using default Marshal serializer (silence with `silence_marshal_warning: true`)
17
+ - Add defense-in-depth input validation for stats command arguments
18
+ - Add `string_fastpath` option to skip serialization for simple strings (byroot)
19
+ - Meta protocol set performance improvement (danmayer)
20
+ - Fix connection_pool 3.0 compatibility for Rack session store
21
+ - Fix session recovery after deletion (stengineering0)
22
+ - Fix cannot read response data included terminator `\r\n` when use meta protocol (matsubara0507)
23
+ - Support SERVER_ERROR response from Memcached as per the [memcached spec](https://github.com/memcached/memcached/blob/e43364402195c8e822bb8f88755a60ab8bbed62a/doc/protocol.txt#L172) (grcooper)
24
+ - Update Socket timeout handling to use Socket#timeout= when available (nickamorim)
25
+ - Serializer: reraise all .load errors as UnmarshalError (olleolleolle)
26
+ - Reconnect gracefully when a fork is detected instead of crashing (PatrickTulskie)
27
+ - Update CI to test against memcached 1.6.40
80
28
 
81
29
  3.2.8
82
30
  ==========
data/Gemfile CHANGED
@@ -5,7 +5,9 @@ source 'https://rubygems.org'
5
5
  gemspec
6
6
 
7
7
  group :development, :test do
8
+ gem 'cgi'
8
9
  gem 'connection_pool'
10
+ gem 'debug' unless RUBY_PLATFORM == 'java'
9
11
  gem 'minitest', '~> 5'
10
12
  gem 'rack', '~> 2.0', '>= 2.2.0'
11
13
  gem 'rake', '~> 13.0'
data/README.md CHANGED
@@ -25,18 +25,6 @@ The name is a variant of Salvador Dali for his famous painting [The Persistence
25
25
  * [Forum](https://github.com/petergoldstein/dalli/discussions/categories/q-a) - If you have questions about Dalli, please post them here.
26
26
  * [Client API](https://www.rubydoc.info/gems/dalli) - Ruby documentation for the `Dalli::Client` API
27
27
 
28
- ## Security Note
29
-
30
- By default, Dalli uses Ruby's Marshal for serialization. Deserializing untrusted data with Marshal can lead to remote code execution. If you cache user-controlled data, or your memcached server could be written to by someone else, consider using a safer serializer:
31
-
32
- ```ruby
33
- Dalli::Client.new('localhost:11211', serializer: Dalli::JSONSerializer)
34
- ```
35
-
36
- `Dalli::JSONSerializer` reads values with `JSON.parse`, so it only ever returns plain hashes, arrays, strings, numbers, booleans and `nil`. Passing the `JSON` module itself (`serializer: JSON`) reads values with `JSON.load`, which can create objects of other classes from a stored `json_class` key and is not recommended for data an attacker might write.
37
-
38
- Per-request `raw: true` returns the stored bytes without deserializing or decompressing them, on every read method that accepts it (`get` and `fetch`).
39
-
40
28
  ## Development
41
29
 
42
30
  After checking out the repo, run `bin/setup` to install dependencies. You can run `bin/console` for an interactive prompt that will allow you to experiment.
data/lib/dalli/client.rb CHANGED
@@ -155,8 +155,8 @@ module Dalli
155
155
  # - nil if the key did not exist.
156
156
  # - false if the value was changed by someone else.
157
157
  # - true if the value was successfully updated.
158
- def cas(key, ttl = nil, req_options = nil, &block)
159
- cas_core(key, false, ttl, req_options, &block)
158
+ def cas(key, ttl = nil, req_options = nil, &)
159
+ cas_core(key, false, ttl, req_options, &)
160
160
  end
161
161
 
162
162
  ##
@@ -166,8 +166,8 @@ module Dalli
166
166
  # Returns:
167
167
  # - false if the value was changed by someone else.
168
168
  # - true if the value was successfully updated.
169
- def cas!(key, ttl = nil, req_options = nil, &block)
170
- cas_core(key, true, ttl, req_options, &block)
169
+ def cas!(key, ttl = nil, req_options = nil, &)
170
+ cas_core(key, true, ttl, req_options, &)
171
171
  end
172
172
 
173
173
  ##
@@ -257,8 +257,6 @@ module Dalli
257
257
  ##
258
258
  # Incr adds the given amount to the counter on the memcached server.
259
259
  # Amt must be a positive integer value.
260
- # Default, if given, must be an Integer (or a String of decimal digits);
261
- # anything else raises ArgumentError.
262
260
  #
263
261
  # If default is nil, the counter must already exist or the operation
264
262
  # will fail and will return nil. Otherwise this method will return
@@ -271,7 +269,6 @@ module Dalli
271
269
  # If the value already exists, it must have been set with raw: true
272
270
  def incr(key, amt = 1, ttl = nil, default = nil)
273
271
  check_positive!(amt)
274
- validate_integer!(:default, default)
275
272
 
276
273
  perform(:incr, key, amt.to_i, ttl_or_default(ttl), default)
277
274
  end
@@ -279,8 +276,6 @@ module Dalli
279
276
  ##
280
277
  # Decr subtracts the given amount from the counter on the memcached server.
281
278
  # Amt must be a positive integer value.
282
- # Default, if given, must be an Integer (or a String of decimal digits);
283
- # anything else raises ArgumentError.
284
279
  #
285
280
  # memcached counters are unsigned and cannot hold negative values. Calling
286
281
  # decr on a counter which is 0 will just return 0.
@@ -296,7 +291,6 @@ module Dalli
296
291
  # If the value already exists, it must have been set with raw: true
297
292
  def decr(key, amt = 1, ttl = nil, default = nil)
298
293
  check_positive!(amt)
299
- validate_integer!(:default, default)
300
294
 
301
295
  perform(:decr, key, amt.to_i, ttl_or_default(ttl), default)
302
296
  end
@@ -379,17 +373,8 @@ module Dalli
379
373
  raise ArgumentError, "Positive values only: #{amt}" if amt.negative?
380
374
  end
381
375
 
382
- # Numeric arguments that become meta protocol flags (GHSA-6wmv-xq9m-fmp7).
383
- # Checked before the request starts so a bad value raises a clean
384
- # ArgumentError, instead of Protocol::Base#request logging it as
385
- # unexpected and closing the connection.
386
- def validate_integer!(name, value)
387
- Dalli::Protocol::Meta::RequestFormatter.integer_flag(name, value) unless value.nil?
388
- end
389
-
390
376
  def cas_core(key, always_set, ttl = nil, req_options = nil)
391
- # req_options reaches the read too, so raw: true returns the stored bytes
392
- (value, cas) = perform(:cas, key, req_options)
377
+ (value, cas) = perform(:cas, key)
393
378
  return if value.nil? && !always_set
394
379
 
395
380
  newvalue = yield(value)
@@ -13,54 +13,8 @@ module Dalli
13
13
  Zlib::Deflate.deflate(data)
14
14
  end
15
15
 
16
- # DEFLATE can't expand data by more than about 1032:1
17
- MAX_EXPANSION = 1032
18
- # Compressed input is fed to zlib this many bytes at a time. MRI's zlib
19
- # yields the output in small chunks; one that returns it instead still
20
- # produces at most MAX_EXPANSION times this per step.
21
- INFLATE_SLICE_BYTES = 64 * 1024
22
-
23
- # max_bytes caps the decompressed size; inflating stops and raises as soon
24
- # as the output passes it, so a small compressed value can't expand into
25
- # gigabytes of memory.
26
- def self.decompress(data, max_bytes: nil)
27
- return Zlib::Inflate.inflate(data) unless may_exceed?(data, max_bytes)
28
-
29
- inflate_within_limit(Zlib::Inflate.new, data, max_bytes)
30
- end
31
-
32
- # False when even the largest possible expansion stays within max_bytes,
33
- # so the value can be inflated in one step
34
- def self.may_exceed?(data, max_bytes)
35
- max_bytes ? data.bytesize * MAX_EXPANSION > max_bytes : false
36
- end
37
-
38
- def self.inflate_within_limit(inflater, data, max_bytes)
39
- out = String.new(encoding: Encoding::BINARY)
40
- 0.step(data.bytesize - 1, INFLATE_SLICE_BYTES) do |pos|
41
- inflate_slice(inflater, data.byteslice(pos, INFLATE_SLICE_BYTES), out, max_bytes)
42
- # Anything after the end of the stream isn't part of the value, as
43
- # with Zlib::Inflate.inflate; finish would return it as output
44
- break if inflater.finished?
45
- end
46
- return out if inflater.finished?
47
-
48
- append_within_limit(out, inflater.finish, max_bytes)
49
- ensure
50
- inflater.close
51
- end
52
-
53
- # Both yielded and returned output count toward the limit
54
- def self.inflate_slice(inflater, slice, out, max_bytes)
55
- rest = inflater.inflate(slice) { |chunk| append_within_limit(out, chunk, max_bytes) }
56
- append_within_limit(out, rest, max_bytes)
57
- end
58
-
59
- def self.append_within_limit(out, chunk, max_bytes)
60
- out << chunk if chunk
61
- return out if out.bytesize <= max_bytes
62
-
63
- raise Dalli::UnmarshalError, "Decompressed value exceeds #{max_bytes} bytes (decompressed_max_bytes)"
16
+ def self.decompress(data)
17
+ Zlib::Inflate.inflate(data)
64
18
  end
65
19
  end
66
20
 
@@ -78,11 +32,9 @@ module Dalli
78
32
  io.string
79
33
  end
80
34
 
81
- def self.decompress(data, max_bytes: nil)
82
- return Zlib::GzipReader.new(StringIO.new(data, 'rb')).read unless Compressor.may_exceed?(data, max_bytes)
83
-
84
- # MAX_WBITS + 16 makes zlib read the gzip format (and check its CRC)
85
- Compressor.inflate_within_limit(Zlib::Inflate.new(Zlib::MAX_WBITS + 16), data, max_bytes)
35
+ def self.decompress(data)
36
+ io = StringIO.new(data, 'rb')
37
+ Zlib::GzipReader.new(io).read
86
38
  end
87
39
  end
88
40
  end
@@ -30,13 +30,10 @@ module Dalli
30
30
 
31
31
  def initialize(client_options)
32
32
  @key_options =
33
- DEFAULTS.merge(client_options.select { |k, _| OPTIONS.include?(k) })
33
+ DEFAULTS.merge(client_options.slice(*OPTIONS))
34
34
  validate_digest_class_option(@key_options)
35
35
 
36
36
  @namespace = namespace_from_options
37
- # The meta protocol base64-encodes keys with whitespace or non-ASCII
38
- # characters, so their length on the wire differs from their length here
39
- @meta_protocol = client_options[:protocol]&.to_s == 'meta'
40
37
  end
41
38
 
42
39
  ##
@@ -54,7 +51,7 @@ module Dalli
54
51
  raise ArgumentError, 'key cannot be blank' unless key&.length&.positive?
55
52
 
56
53
  key = key_with_namespace(key)
57
- wire_length(key) > MAX_KEY_LENGTH ? truncated_key(key) : key
54
+ key.length > MAX_KEY_LENGTH ? truncated_key(key) : key
58
55
  end
59
56
 
60
57
  ##
@@ -80,7 +77,7 @@ module Dalli
80
77
  def namespace_regexp
81
78
  return /\A#{Regexp.escape(evaluate_namespace)}:/ if namespace.is_a?(Proc)
82
79
 
83
- @namespace_regexp ||= /\A#{Regexp.escape(namespace)}:/.freeze unless namespace.nil?
80
+ @namespace_regexp ||= /\A#{Regexp.escape(namespace)}:/ unless namespace.nil?
84
81
  end
85
82
 
86
83
  def validate_digest_class_option(opts)
@@ -110,30 +107,7 @@ module Dalli
110
107
  ##
111
108
  def truncated_key(key)
112
109
  digest = digest_class.hexdigest(key)
113
- prefix = key[0, [prefix_length(digest), 0].max]
114
- truncated = "#{prefix}#{TRUNCATED_KEY_SEPARATOR}#{digest}"
115
- # A prefix with whitespace or non-ASCII characters sends the whole key
116
- # base64-encoded, which can still be too long on the wire; shorten it.
117
- while wire_length(truncated) > MAX_KEY_LENGTH && !prefix.empty?
118
- prefix = prefix[0...-1]
119
- truncated = "#{prefix}#{TRUNCATED_KEY_SEPARATOR}#{digest}"
120
- end
121
- return truncated if wire_length(truncated) <= MAX_KEY_LENGTH
122
-
123
- # Only possible with a digest_class whose digests aren't short hex strings
124
- raise ArgumentError, "#{digest_class} digests are too long to fit in a truncated key"
125
- end
126
-
127
- # memcached's key limit applies to the key as sent: in bytes, and, with
128
- # the meta protocol, after base64 encoding when the key needs it. A key
129
- # within the limit in characters can still be too long. With the meta
130
- # protocol memcached rejects it with a CLIENT_ERROR that must never reach
131
- # a pipelined get; with the binary protocol it drops the connection.
132
- def wire_length(key)
133
- return key.bytesize unless @meta_protocol
134
-
135
- encoded, = Dalli::Protocol::Meta::KeyRegularizer.encode(key)
136
- encoded.bytesize
110
+ "#{key[0, prefix_length(digest)]}#{TRUNCATED_KEY_SEPARATOR}#{digest}"
137
111
  end
138
112
 
139
113
  def prefix_length(digest)
@@ -27,12 +27,6 @@ module Dalli
27
27
  @connection_manager = ConnectionManager.new(hostname, port, socket_type, @options)
28
28
  end
29
29
 
30
- # True when this request passed raw: true. A stored value is then
31
- # returned as stored, never deserialized or decompressed.
32
- def raw_request?(options)
33
- options.is_a?(Hash) && options[:raw] ? true : false
34
- end
35
-
36
30
  # Chokepoint method for error handling and ensuring liveness
37
31
  def request(opkey, *args)
38
32
  verify_state(opkey)
@@ -124,16 +118,13 @@ module Dalli
124
118
  def pipeline_abort
125
119
  response_buffer.clear
126
120
  @connection_manager.abort_request!
127
- return unless connected?
128
-
129
- # Closes the connection, which ensures that our connection is in a
130
- # clean state for future requests. The get_multi ran out of time
131
- # overall rather than hitting a socket error, so it doesn't count
132
- # toward socket_max_failures: two slow get_multis in a row would
133
- # otherwise mark a healthy server down.
134
- Dalli.logger.warn { "#{name} didn't finish a get_multi within socket_timeout; closing the connection" }
135
- @connection_manager.close
136
- nil
121
+ return true unless connected?
122
+
123
+ # Closes the connection, which ensures that our connection
124
+ # is in a clean state for future requests
125
+ @connection_manager.error_on_request!('External timeout')
126
+ rescue NetworkError
127
+ true
137
128
  end
138
129
 
139
130
  # Did the last call to #pipeline_response_setup complete successfully?
@@ -163,6 +154,8 @@ module Dalli
163
154
  private
164
155
 
165
156
  ALLOWED_QUIET_OPS = %i[add replace set delete incr decr append prepend flush noop].freeze
157
+ private_constant :ALLOWED_QUIET_OPS
158
+
166
159
  def verify_allowed_quiet!(opkey)
167
160
  return if ALLOWED_QUIET_OPS.include?(opkey)
168
161
 
@@ -66,9 +66,7 @@ module Dalli
66
66
  raise Dalli::DalliError, "Response error #{resp_header.status}: #{RESPONSE_CODES[resp_header.status]}"
67
67
  end
68
68
 
69
- # raw: true returns the value as stored, without deserializing or
70
- # decompressing it, whatever flags it was stored with
71
- def get(cache_nils: false, raw: false)
69
+ def get(cache_nils: false)
72
70
  resp_header, body = read_response
73
71
 
74
72
  return false if resp_header.not_stored? # Not stored, normal status for add operation
@@ -77,7 +75,7 @@ module Dalli
77
75
  raise_on_not_ok!(resp_header)
78
76
  return true unless body
79
77
 
80
- unpack_response_body(resp_header, body, !raw).last
78
+ unpack_response_body(resp_header, body, true).last
81
79
  end
82
80
 
83
81
  ##
@@ -102,8 +100,7 @@ module Dalli
102
100
  true
103
101
  end
104
102
 
105
- # raw: true returns the value as stored; see #get
106
- def data_cas_response(raw: false)
103
+ def data_cas_response
107
104
  resp_header, body = read_response
108
105
  return [nil, resp_header.cas] if resp_header.not_found?
109
106
  return [nil, false] if resp_header.not_stored?
@@ -111,7 +108,7 @@ module Dalli
111
108
  raise_on_not_ok!(resp_header)
112
109
  return [nil, resp_header.cas] unless body
113
110
 
114
- [unpack_response_body(resp_header, body, !raw).last, resp_header.cas]
111
+ [unpack_response_body(resp_header, body, true).last, resp_header.cas]
115
112
  end
116
113
 
117
114
  # Returns the new value for the key, if found and updated
@@ -212,17 +209,6 @@ module Dalli
212
209
  # The remaining three values in the array are the ResponseHeader,
213
210
  # key, and value.
214
211
  ##
215
- # memcached can't store an item larger than 1 GiB (its -I maximum). A
216
- # pipelined reply claiming an impossible size would otherwise have the
217
- # buffer wait for (and accumulate) that many bytes.
218
- MAX_BODY_BYTES = 1024 * 1024 * 1024
219
-
220
- def check_body_size!(size)
221
- return if size <= MAX_BODY_BYTES
222
-
223
- raise Dalli::DalliError, "Reply body size #{size} is out of range"
224
- end
225
-
226
212
  def getk_response_from_buffer(buf)
227
213
  # There's no header in the buffer, so don't advance
228
214
  return [0, nil, nil, nil, nil] unless contains_header?(buf)
@@ -236,8 +222,6 @@ module Dalli
236
222
  # error response that needs to be discarded.
237
223
  return [ResponseHeader::SIZE, resp_header.ok?, resp_header.cas, nil, nil] if body_len.zero?
238
224
 
239
- check_body_size!(body_len)
240
-
241
225
  resp_size = ResponseHeader::SIZE + body_len
242
226
  # The header is in the buffer, but the body is not. As we don't have
243
227
  # a complete response, don't advance the buffer
@@ -22,7 +22,7 @@ module Dalli
22
22
  def get(key, options = nil)
23
23
  req = RequestFormatter.standard_request(opkey: :get, key: key)
24
24
  write(req)
25
- response_processor.get(cache_nils: cache_nils?(options), raw: raw_request?(options))
25
+ response_processor.get(cache_nils: cache_nils?(options))
26
26
  end
27
27
 
28
28
  def quiet_get_request(key)
@@ -33,7 +33,7 @@ module Dalli
33
33
  ttl = TtlSanitizer.sanitize(ttl)
34
34
  req = RequestFormatter.standard_request(opkey: :gat, key: key, ttl: ttl)
35
35
  write(req)
36
- response_processor.get(cache_nils: cache_nils?(options), raw: raw_request?(options))
36
+ response_processor.get(cache_nils: cache_nils?(options))
37
37
  end
38
38
 
39
39
  def touch(key, ttl)
@@ -44,10 +44,10 @@ module Dalli
44
44
 
45
45
  # TODO: This is confusing, as there's a cas command in memcached
46
46
  # and this isn't it. Maybe rename? Maybe eliminate?
47
- def cas(key, options = nil)
47
+ def cas(key)
48
48
  req = RequestFormatter.standard_request(opkey: :get, key: key)
49
49
  write(req)
50
- response_processor.data_cas_response(raw: raw_request?(options))
50
+ response_processor.data_cas_response
51
51
  end
52
52
 
53
53
  # Storage Commands
@@ -119,6 +119,7 @@ module Dalli
119
119
  # if the key doesn't already exist, rather than
120
120
  # setting the initial value
121
121
  NOT_FOUND_EXPIRY = 0xFFFFFFFF
122
+ private_constant :NOT_FOUND_EXPIRY
122
123
 
123
124
  def decr_incr(opkey, key, count, ttl, initial)
124
125
  expiry = initial ? TtlSanitizer.sanitize(ttl) : NOT_FOUND_EXPIRY
@@ -38,7 +38,6 @@ module Dalli
38
38
  @sock = nil
39
39
  @pid = nil
40
40
 
41
- @fail_count = 0
42
41
  reset_down_info
43
42
  end
44
43
 
@@ -54,7 +53,7 @@ module Dalli
54
53
  Dalli.logger.debug { "Dalli::Server#connect #{name}" }
55
54
 
56
55
  @sock = memcached_socket
57
- @pid = Process.pid
56
+ @pid = PIDCache.pid
58
57
  @request_in_progress = false
59
58
  rescue SystemCallError, *TIMEOUT_ERRORS, EOFError, SocketError => e
60
59
  # SocketError = DNS resolution failure
@@ -85,8 +84,6 @@ module Dalli
85
84
  def down!
86
85
  close
87
86
  log_down_detected
88
- # Once down_retry_delay passes, the server gets a full set of attempts
89
- @fail_count = 0
90
87
 
91
88
  @error = $ERROR_INFO&.class&.name
92
89
  @msg ||= $ERROR_INFO&.message
@@ -103,20 +100,20 @@ module Dalli
103
100
 
104
101
  def confirm_ready!
105
102
  close if request_in_progress?
106
- close_on_fork if fork_detected?
103
+ reconnect_on_fork if fork_detected?
107
104
  end
108
105
 
109
106
  def confirm_in_progress!
110
107
  raise '[Dalli] No request in progress. This may be a bug in Dalli.' unless request_in_progress?
111
108
 
112
- close_on_fork if fork_detected?
109
+ reconnect_on_fork if fork_detected?
113
110
  end
114
111
 
115
112
  def close
116
113
  return unless @sock
117
114
 
118
115
  begin
119
- close_socket
116
+ @sock.close
120
117
  rescue StandardError
121
118
  nil
122
119
  end
@@ -125,13 +122,6 @@ module Dalli
125
122
  abort_request!
126
123
  end
127
124
 
128
- # A forked child shares the parent's connection, so it closes only its
129
- # own file descriptor. Closing the TLS socket itself would send
130
- # close_notify and end the parent's TLS session.
131
- def close_socket
132
- fork_detected? ? @sock.to_io.close : @sock.close
133
- end
134
-
135
125
  def connected?
136
126
  !@sock.nil?
137
127
  end
@@ -150,11 +140,6 @@ module Dalli
150
140
  raise '[Dalli] No request in progress. This may be a bug in Dalli.' unless @request_in_progress
151
141
 
152
142
  @request_in_progress = false
153
- # A completed request is what proves the server healthy again, so the
154
- # failure count resets here rather than on reconnect: a server that
155
- # accepts connections but never answers would otherwise reset it on
156
- # every retry and never reach socket_max_failures.
157
- @fail_count = 0
158
143
  end
159
144
 
160
145
  def abort_request!
@@ -169,19 +154,7 @@ module Dalli
169
154
  error_on_request!(e)
170
155
  end
171
156
 
172
- # memcached can't store an item larger than 1 GiB (its -I maximum), so a
173
- # reply claiming more (or a negative size) is malformed or hostile.
174
- # Reading allocates the full count up front, so check before reading.
175
- MAX_READ_BYTES = (1024 * 1024 * 1024) + 2 # plus the value's trailing "\r\n"
176
-
177
- def check_read_size!(count)
178
- return if count.between?(0, MAX_READ_BYTES)
179
-
180
- raise Dalli::DalliError, "Reply size #{count} from #{name} is out of range"
181
- end
182
-
183
157
  def read(count)
184
- check_read_size!(count)
185
158
  @sock.readfull(count)
186
159
  rescue SystemCallError, *TIMEOUT_ERRORS, EOFError => e
187
160
  error_on_request!(e)
@@ -222,9 +195,8 @@ module Dalli
222
195
  raise Dalli::NetworkError, message
223
196
  end
224
197
 
225
- # Called on connect. Deliberately leaves @fail_count alone; see
226
- # finish_request!.
227
198
  def reset_down_info
199
+ @fail_count = 0
228
200
  @down_at = nil
229
201
  @last_down_at = nil
230
202
  @msg = nil
@@ -240,28 +212,22 @@ module Dalli
240
212
  end
241
213
 
242
214
  def log_warn_message(err_or_string)
243
- detail = err_or_string.is_a?(String) ? err_or_string : "#{err_or_string.class}: #{err_or_string.message}"
244
215
  Dalli.logger.warn do
245
216
  detail = err_or_string.is_a?(String) ? err_or_string : "#{err_or_string.class}: #{err_or_string.message}"
246
217
  "#{name} failed (count: #{@fail_count}) #{detail}"
247
218
  end
248
219
  end
249
220
 
250
- def close_on_fork
221
+ def reconnect_on_fork
251
222
  message = 'Fork detected, re-connecting child process...'
252
223
  Dalli.logger.info { message }
253
- # Closes the inherited socket without touching the parent's
254
- # connection, setting us up for reconnect on next request.
224
+ # Close socket on a fork and reconnect immediately
255
225
  close
256
- raise Dalli::NetworkError, message
226
+ establish_connection
257
227
  end
258
228
 
259
229
  def fork_detected?
260
- # Process.pid rather than PIDCache: PIDCache is refreshed by a
261
- # Process._fork hook, and another library's fork hook (such as
262
- # connection_pool closing its connections) can run in the child before
263
- # it, which would close the parent's TLS session as if it were ours.
264
- @pid && @pid != Process.pid
230
+ @pid && @pid != PIDCache.pid
265
231
  end
266
232
 
267
233
  def log_down_detected
@@ -10,7 +10,7 @@ module Dalli
10
10
  # memcached supports the use of base64 hashes for keys containing
11
11
  # whitespace or non-ASCII characters, provided the 'b' flag is included in the request.
12
12
  class KeyRegularizer
13
- WHITESPACE = /\s/.freeze
13
+ WHITESPACE = /\s/
14
14
 
15
15
  def self.encode(key)
16
16
  return [key, false] if key.ascii_only? && !WHITESPACE.match(key)