dalli 5.2.2 → 5.2.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: cd910ad2e0c6e5aa47b8591e9ba8c5806b6208fbab0ae0852637aecc48939051
4
- data.tar.gz: bd0db96211060011acb2135feb337d8acc88638b08c57b3fa875445695a09b59
3
+ metadata.gz: 1a31a4159ee5cd637dca6676f343595f653798864bef28b109fbc329debe2aff
4
+ data.tar.gz: efba0a1a4bb20eff2fd4f6b26c15359351484401502a798fa398b5bf1301d046
5
5
  SHA512:
6
- metadata.gz: 82da7aebd50661e1ab9af25dab7d66e5f70cb15260ff9ca4b3476e63eeb60cbb2ad1299718e56312f9a6a61d7a2b14d394fcc470325329f1152db42ac921ee6d
7
- data.tar.gz: a036421fac121aa342adc3beced711ce64a1bd241673a9314f1f1454a951515890732c4c9651e52baf65c94cd01c4b5cbb655c8dcca4fa4db03d656c3cfc4abe
6
+ metadata.gz: 38f6d3ba18790b94875f18bf6239a520f57144ba763826063471bea2047aa6def2cc528d37de35f29037a0c33ca9bb8b46d7ed0ddc5dccebf8914888c71dafd7
7
+ data.tar.gz: 9f091240e20313b7cbb9250e900cbf3aa02f9aedbffceabeedc7355bc780cd3895e231c77bb091ab97680dbdac85e093635e3cfbca904fd7f095512181e4a59d
data/CHANGELOG.md CHANGED
@@ -4,6 +4,58 @@ Dalli Changelog
4
4
  Unreleased
5
5
  ==========
6
6
 
7
+ 5.2.3
8
+ ==========
9
+
10
+ Security:
11
+
12
+ - Don't rely on memcached's auto-create for keys that need base64 encoding (GHSA-r47x-6p6r-mvcp)
13
+ - memcached stores an item made by auto-create (`ma` with `N`/`J`, or `mg` with `N`) for a base64-encoded key without its binary-key flag, and later replies name the item by its raw key bytes. A key containing spaces or CR/LF could then make a multi-get read the replies after it as different ones
14
+ - `incr` and `decr` with an initial value now create such counters with an add instead. Return values are unchanged
15
+ - The new `verify_reply_keys` option makes `get_multi`, `get_multi_cas` and `get_multi_with_metadata` ignore replies for keys they didn't request, which guards against items created by older Dalli versions or other clients until memcached is upgraded. It's off by default, since it costs a multi-get about 15%
16
+ - Affects the meta protocol since 3.2.0; fixed in 5.2.3, 5.1.5, 5.0.11, 4.3.8 and 3.2.14. memcached 1.6.45 and earlier still name these items by their raw key bytes
17
+
18
+ Bug fixes:
19
+
20
+ - `get_multi_with_metadata` and its `CacheResult` adapter now retry transient network errors and raise `Dalli::NetworkError` on terminal failures instead of returning misses or partial results (#1222, @jaideeppyne)
21
+ - A per-request `raw: true` write no longer compresses the value (#1213)
22
+ - Values at or above the compression threshold (4 KB by default) were compressed and flagged even with `raw: true`. Raw reads ignore flags, so reading them back with `raw: true` returned the compressed bytes
23
+ - Raw values are now stored exactly as given. Callers that pass `raw: true` without `compress: false` store large values uncompressed; Rails' `MemCacheStore` passes `compress: false` and isn't affected
24
+ - Fix `cas` and `cas!` edge cases, and make `flush` attempt every server (#1214)
25
+ - `cas!` on a missing key used an unconditional set, so a value another client created while the block ran was overwritten. It now adds the key and returns `false` if another client got there first
26
+ - With `cache_nils: true`, `cas` treated a stored `nil` as a miss and didn't call the block
27
+ - The docs now say what `cas` and `cas!` return on success: the new CAS value, not `true`. The return values are unchanged
28
+ - `flush` stopped at the first server that raised, so later servers weren't flushed. It now flushes every server it can reach, then raises the first error
29
+ - Handle failed TLS handshakes like other connection failures, and warn when TLS verification is off (#1215)
30
+ - A failed or timed-out handshake left the TCP socket open until GC and raised `OpenSSL::SSL::SSLError` directly, so the server was never marked down. The socket is now closed, and the error goes through the usual retry and `socket_max_failures` handling, raising `Dalli::NetworkError` (or `Dalli::RingError` once no server is available)
31
+ - Certificate and hostname checks come only from the `ssl_context` you pass, and a bare `OpenSSL::SSL::SSLContext.new` checks neither. Dalli now logs a warning once per process when `verify_mode` doesn't include `VERIFY_PEER` or `verify_hostname` is false; see the new TLS section in the README
32
+ - Connection and configuration fixes (#1216)
33
+ - `memcached://` URIs with a bracketed IPv6 host (`memcached://[::1]:11211`) now connect
34
+ - A server weight below 1 raises `ArgumentError`. Before, a ring where every weight was 0 raised `FloatDomainError`
35
+ - A reply line longer than 8 KiB without a CRLF, or one cut short by the connection closing, is now a network error that closes the connection. Before, the line had no length limit
36
+ - On CRuby, DNS resolution is now limited by `socket_timeout` (`resolv_timeout:`) as well as the connect
37
+ - `Dalli::Client.new` no longer modifies the options hash it's given, and accepts a frozen one (#1217)
38
+ - `value_max_bytes` now counts the key and memcached's per-item overhead (#1217)
39
+ - memcached's `-I` limit covers the whole item, not only the value. A value of exactly 1 MB passed Dalli's check and then failed in memcached with `Dalli::ServerError`, which `quiet` blocks and `set_multi` silently discarded
40
+ - Dalli now checks value + key + 63 bytes, the overhead measured on memcached 1.6, and raises `Dalli::ValueOverMaxSize` before sending. A custom `value_max_bytes` is now that much stricter
41
+ - A String `value_max_bytes` is converted to an Integer instead of raising on every set
42
+ - `Rack::Session::Dalli` fixes (#1217)
43
+ - It no longer requires the `connection_pool` gem unless `pool_size` or `pool_timeout` is given
44
+ - `pool_size` and `pool_timeout` are no longer passed to `Dalli::Client`, and the options hash isn't modified
45
+ - A failed session delete is logged as a warning, not only when `$VERBOSE` is set
46
+ - One value that can't be stored no longer discards other buffered writes (#1218)
47
+ - A value that raised `Dalli::MarshalError` or `Dalli::ValueOverMaxSize` closed the connection, which dropped writes still buffered from the same `quiet` block or `set_multi`. Multi-server `set_multi` did this without raising
48
+ - Both `set_multi` paths now store every other key, skip the bad one, and then raise. Single-server `set_multi` used to store nothing
49
+ - When single-server `set_multi` failed to marshal a value, the error log included every value in the batch. It now lists only the keys
50
+ - `Rack::Session::Dalli`'s 401 for a session deleted mid-request is now a valid Rack response (#1220)
51
+ - When a session is deleted while a request is using it (for example, by a logout in another request), the store still returns 401 instead of writing the session back. The 401 now has a `content-type` and expires the session cookie, so the browser stops sending the dead session ID
52
+ - The app's response body is now closed, as the Rack spec requires. Before, it was dropped without `close`
53
+ - The app has already run when the 401 is returned, so its side effects have happened. This is documented in the README
54
+
55
+ Improvements:
56
+
57
+ - Add a `connect_timeout` option to set a separate timeout for DNS resolution and TCP/UNIX socket connects (including reconnects); it defaults to `socket_timeout`. TLS handshakes continue to use `socket_timeout` (#1221, @underwoo16)
58
+
7
59
  5.2.2
8
60
  ==========
9
61
 
data/README.md CHANGED
@@ -59,6 +59,17 @@ Dalli::Client.new('localhost:11211', namespace: 'myapp', namespace_separator: '/
59
59
 
60
60
  The separator must be a single non-alphanumeric character. Valid examples: `:`, `/`, `|`, `.`, `-`, `_`, `#`
61
61
 
62
+ ### Maximum Item Size
63
+
64
+ memcached rejects items larger than its `-I` setting (1MB by default). Set `value_max_bytes` to the same value if you change it:
65
+
66
+ ```ruby
67
+ # memcached started with -I 2m
68
+ Dalli::Client.new('localhost:11211', value_max_bytes: 2 * 1024 * 1024)
69
+ ```
70
+
71
+ memcached's limit covers the whole item, so Dalli checks the stored (serialized and compressed) value plus the key plus 63 bytes of per-item overhead against `value_max_bytes`, and raises `Dalli::ValueOverMaxSize` before sending anything larger. With the defaults, the largest value you can store under a 10-byte key is 1,048,576 - 10 - 63 = 1,048,503 bytes.
72
+
62
73
  ### Deferred Draining
63
74
 
64
75
  By default, a `quiet` (or `multi`) block ends by waiting for the replies memcached still sends for its quiet requests, one round trip for each server the block wrote to. With `defer_drain: true`, the block returns without waiting:
@@ -112,6 +123,35 @@ The digest is MD5 by default. `digest_class:` takes any object that responds to
112
123
 
113
124
  See the [5.0-Upgrade.md](5.0-Upgrade.md) guide for upgrade information.
114
125
 
126
+ ### Counters and verify_reply_keys
127
+
128
+ memcached names an item made by auto-create for a base64-encoded key (one with whitespace, control or non-ASCII characters) by its raw key bytes in later replies. Since 5.2.3, Dalli creates counters for such keys with an add rather than auto-create. Items created before then, or by other clients, can still be named that way until memcached releases a fix (1.6.45 and earlier behave this way). If keys passed to `incr` or `decr` with an initial value can contain untrusted input, set `verify_reply_keys: true` until then: multi-gets then ignore replies for keys they didn't request, at a cost of about 15% per multi-get.
129
+
130
+ ## TLS
131
+
132
+ Pass an `OpenSSL::SSL::SSLContext` as `ssl_context:` to connect over TLS. Certificate and hostname verification come only from that context: Dalli adds no checks of its own. A bare `OpenSSL::SSL::SSLContext.new` verifies nothing, so it accepts any certificate from any server. Configure the context to verify both:
133
+
134
+ ```ruby
135
+ ssl_context = OpenSSL::SSL::SSLContext.new
136
+ ssl_context.set_params(
137
+ verify_mode: OpenSSL::SSL::VERIFY_PEER,
138
+ verify_hostname: true,
139
+ ca_file: '/path/to/ca.crt' # omit to use the system's trusted certificates
140
+ )
141
+
142
+ Dalli::Client.new('memcached.example.com:11211', ssl_context: ssl_context)
143
+ ```
144
+
145
+ Dalli logs a warning once per process when the context's `verify_mode` is `VERIFY_NONE`, or when it is `VERIFY_PEER` but `verify_hostname` is false.
146
+
147
+ A failed TLS handshake counts as a failed connection attempt, like a refused connection: once `socket_max_failures` attempts have failed, the server is marked down and requests fail over to the other servers.
148
+
149
+ ## Rack Sessions
150
+
151
+ `Rack::Session::Dalli` stores Rack sessions in memcached. If a session is deleted while a request is using it (for example, when the user logs out in another tab during a slow request), the store doesn't write the session back, so a logged-out session can't come back. It replaces the app's response with a `401 Wrong session ID` that expires the session cookie.
152
+
153
+ The app has already run by then, so whatever it did (database writes, emails, and so on) has happened, and only its response is lost. If a request needs a different outcome, check that the session still exists before making the changes.
154
+
115
155
  ## OpenTelemetry Tracing
116
156
 
117
157
  Dalli automatically instruments operations with [OpenTelemetry](https://opentelemetry.io/) when the SDK is present. No configuration is required - just add the OpenTelemetry gems to your application:
data/lib/dalli/client.rb CHANGED
@@ -35,15 +35,20 @@ module Dalli
35
35
  # to memcached. Default: true.
36
36
  # - :compression_min_size - the minimum size (in bytes) for which Dalli will compress values sent to Memcached.
37
37
  # Defaults to 4K.
38
+ # - :value_max_bytes - the largest item memcached will accept, i.e. its -I setting. Defaults to 1MB. Like
39
+ # memcached, Dalli counts the key and 63 bytes of item overhead against this limit along
40
+ # with the (serialized, compressed) value, so the largest value that can be stored is
41
+ # value_max_bytes - key bytesize - 63. Larger values raise Dalli::ValueOverMaxSize.
38
42
  # - :serializer - defaults to Marshal
39
43
  # - :compressor - defaults to Dalli::Compressor, a Zlib-based implementation
40
44
  # - :cache_nils - defaults to false, if true Dalli will not treat cached nil values as 'not found' for
41
45
  # #fetch operations.
42
46
  # - :raw - If set, disables serialization and compression entirely at the client level.
43
47
  # Only String values are supported. This is useful when the caller handles its own
44
- # serialization (e.g., Rails' ActiveSupport::Cache). Note: this is different from
45
- # the per-request :raw option which converts values to strings but still uses the
46
- # serialization pipeline.
48
+ # serialization (e.g., Rails' ActiveSupport::Cache). The per-request :raw option
49
+ # (e.g. set(key, value, ttl, raw: true)) does the same for a single call: String
50
+ # values are stored as-is, uncompressed, and non-String values raise
51
+ # Dalli::MarshalError.
47
52
  # - :digest_class - defaults to Digest::MD5, allows you to pass in an object that responds to the hexdigest method,
48
53
  # useful for injecting a FIPS compliant hash object.
49
54
  # - :otel_db_statement - controls the +db.query.text+ span attribute when OpenTelemetry is loaded.
@@ -58,6 +63,17 @@ module Dalli
58
63
  # quiet requests (NF, NS, errors) wait on the connection until then, so a client that only makes
59
64
  # quiet requests should call #drain_deferred_responses periodically. See "Deferred Draining" in
60
65
  # the README.
66
+ # - :socket_timeout - timeout in seconds for reads and writes on a connection, and for the TLS
67
+ # handshake. Also used for connecting unless :connect_timeout is set.
68
+ # Default: 1.
69
+ # - :connect_timeout - timeout in seconds for DNS resolution and TCP/UNIX socket connects,
70
+ # including reconnects after a failure. Defaults to :socket_timeout.
71
+ # TLS handshakes use :socket_timeout.
72
+ # - :verify_reply_keys - defaults to false. When true, get_multi, get_multi_cas and get_multi_with_metadata only
73
+ # accept replies for the keys they requested; after a reply naming another key, the rest of
74
+ # that server's replies are treated as misses. memcached can name an item by its raw key
75
+ # bytes when the item was created by auto-create from a base64-encoded key, which Dalli no
76
+ # longer does. Costs a multi-get about 15%.
61
77
  #
62
78
  def initialize(servers = nil, options = {})
63
79
  @normalized_servers = ::Dalli::ServersArgNormalizer.normalize_servers(servers)
@@ -226,9 +242,14 @@ module Dalli
226
242
  #
227
243
  # `req_options` accepts :p_token/:l_token, applied to every key in the batch.
228
244
  #
245
+ # A transient network error is retried automatically. If a server remains
246
+ # unreachable after retrying, raises Dalli::NetworkError rather than
247
+ # returning its keys as misses or yielding a partial result.
248
+ #
229
249
  # @param keys [Array<String>] the keys to fetch
230
250
  # @param req_options [Hash, nil] routing-token options
231
251
  # @return [Hash] key => { value:, cas:, stale:, miss: }
252
+ # @raise [Dalli::NetworkError] if a server is unreachable after retrying
232
253
  def get_multi_with_metadata(*keys, req_options: nil, &block)
233
254
  keys.flatten!
234
255
  keys.compact!
@@ -371,24 +392,26 @@ module Dalli
371
392
  # compare and swap values using optimistic locking.
372
393
  # Fetch the existing value for key.
373
394
  # If it exists, yield the value to the block.
374
- # Add the block's return value as the new value for the key.
375
- # Add will fail if someone else changed the value.
395
+ # Store the block's return value as the new value for the key.
396
+ # The store will fail if someone else changed the value.
376
397
  #
377
398
  # Returns:
378
- # - nil if the key did not exist.
399
+ # - nil if the key did not exist (or held nil and :cache_nils is false);
400
+ # the block is not called.
379
401
  # - false if the value was changed by someone else.
380
- # - true if the value was successfully updated.
402
+ # - the new CAS value (a positive Integer) if the value was successfully updated.
381
403
  def cas(key, ttl = nil, req_options = nil, &)
382
404
  cas_core(key, false, ttl, req_options, &)
383
405
  end
384
406
 
385
407
  ##
386
408
  # like #cas, but will yield to the block whether or not the value
387
- # already exists.
409
+ # already exists. If the key did not exist, the block receives nil.
388
410
  #
389
411
  # Returns:
390
- # - false if the value was changed by someone else.
391
- # - true if the value was successfully updated.
412
+ # - false if the value was changed by someone else, including another
413
+ # client creating a missing key while the block ran.
414
+ # - the new CAS value (a positive Integer) if the value was successfully updated.
392
415
  def cas!(key, ttl = nil, req_options = nil, &)
393
416
  cas_core(key, true, ttl, req_options, &)
394
417
  end
@@ -650,9 +673,20 @@ module Dalli
650
673
  ##
651
674
  # Flush the memcached server, at 'delay' seconds in the future.
652
675
  # Delay defaults to zero seconds, which means an immediate flush.
676
+ # Every server is attempted. If any of them fails, the first error is
677
+ # raised once all the others have been flushed.
653
678
  ##
654
679
  def flush(delay = 0)
655
- ring.servers.map { |s| s.request(:flush, delay) }
680
+ first_error = nil
681
+ results = ring.servers.map do |s|
682
+ s.request(:flush, delay)
683
+ rescue StandardError => e
684
+ first_error ||= e
685
+ nil
686
+ end
687
+ raise first_error if first_error
688
+
689
+ results
656
690
  end
657
691
  alias flush_all flush
658
692
 
@@ -805,16 +839,26 @@ module Dalli
805
839
  Dalli.logger.debug { e.inspect }
806
840
  Dalli.logger.debug { 'retrying single-server get_multi because of network error' }
807
841
  retry
842
+ rescue Dalli::UnexpectedReplyKeyError => e
843
+ # The connection is closed and the keys are treated as missing, as the
844
+ # pipelined get_multi does for a server whose replies are out of step
845
+ Dalli.logger.warn { e.message }
846
+ {}
808
847
  end
809
848
 
810
849
  def single_server_get_multi_with_metadata(keys, req_options = nil)
811
- keys.map! { |k| @key_manager.validate_key(k.to_s) }
850
+ validated_keys = keys.map { |k| @key_manager.validate_key(k.to_s) }
812
851
  return {} unless (server = single_server)
813
852
 
814
- result = server.request(:read_multi_with_metadata_req, keys, req_options)
853
+ result = server.request(:read_multi_with_metadata_req, validated_keys, req_options)
815
854
  result.transform_keys! { |k| @key_manager.key_without_namespace(k) }
816
855
  result
817
- rescue Dalli::NetworkError
856
+ rescue Dalli::RetryableNetworkError => e
857
+ Dalli.logger.debug { e.inspect }
858
+ Dalli.logger.debug { 'retrying single-server get_multi_with_metadata because of network error' }
859
+ retry
860
+ rescue Dalli::UnexpectedReplyKeyError => e
861
+ Dalli.logger.warn { e.message }
818
862
  {}
819
863
  end
820
864
 
@@ -913,12 +957,23 @@ module Dalli
913
957
  def cas_core(key, always_set, ttl = nil, req_options = nil)
914
958
  validate_routing_tokens!(req_options)
915
959
  (value, cas) = perform(:cas, key, req_options)
916
- return if value.nil? && !always_set
960
+ return if !always_set && cas_miss?(value, cas)
917
961
 
918
962
  newvalue = yield(value)
963
+ # A CAS of 0 means the key was missing. A set with CAS 0 is unconditional,
964
+ # so add instead: it fails if another client created the key meanwhile.
965
+ return perform(:add, key, newvalue, ttl_or_default(ttl), req_options) if cas.zero?
966
+
919
967
  perform(:set, key, newvalue, ttl_or_default(ttl), cas, req_options)
920
968
  end
921
969
 
970
+ # The CAS lookup returns [nil, 0] on a miss rather than NOT_FOUND, and a
971
+ # stored item always has a non-zero CAS. So under cache_nils a zero CAS
972
+ # marks the miss, and a stored nil is a hit, as in #not_found?.
973
+ def cas_miss?(value, cas)
974
+ cache_nils ? cas.zero? : value.nil?
975
+ end
976
+
922
977
  def fetch_with_lock_request(key, ttl, lock_ttl, recache_threshold, req_options)
923
978
  server = ring.server_for_key(key)
924
979
  # req_options is the base, not the override: fetch_with_lock's own
@@ -986,7 +1041,10 @@ module Dalli
986
1041
  retry
987
1042
  end
988
1043
 
989
- def normalize_options(opts)
1044
+ # Works on a copy so the caller's hash is neither modified here nor able
1045
+ # to change this client's configuration later.
1046
+ def normalize_options(options)
1047
+ opts = options.dup
990
1048
  opts[:expires_in] = opts[:expires_in].to_i if opts[:expires_in]
991
1049
  opts
992
1050
  rescue NoMethodError
@@ -60,6 +60,9 @@ module Dalli
60
60
  #
61
61
  # `req_options` accepts :p_token/:l_token, applied to every key in the batch.
62
62
  #
63
+ # A transient network error is retried automatically. A terminal
64
+ # Dalli::NetworkError propagates instead of returning partial metadata.
65
+ #
63
66
  def process_with_metadata(keys, req_options = nil)
64
67
  return {} if keys.empty?
65
68
 
@@ -67,9 +70,9 @@ module Dalli
67
70
  results = {}
68
71
  groups_for_keys(keys).each do |server, keys_for_server|
69
72
  results.merge!(server.request(:read_multi_with_metadata_req, keys_for_server, req_options))
70
- rescue Dalli::RetryableNetworkError
73
+ rescue Dalli::NetworkError
71
74
  raise
72
- rescue DalliError, NetworkError => e
75
+ rescue DalliError => e
73
76
  Dalli.logger.debug { e.inspect }
74
77
  Dalli.logger.debug { "unable to get keys for server #{server.name}" }
75
78
  end
@@ -18,6 +18,9 @@ module Dalli
18
18
  # A transient network error is retried automatically. If a server remains
19
19
  # unreachable after retrying, raises Dalli::NetworkError.
20
20
  #
21
+ # A value that can't be marshalled or is over value_max_bytes is skipped;
22
+ # the other keys are still stored, then the first such error is raised.
23
+ #
21
24
  # @param hash [Hash] key-value pairs to set
22
25
  # @param ttl [Integer] time-to-live in seconds
23
26
  # @param req_options [Hash] options passed to each set operation
@@ -26,10 +29,12 @@ module Dalli
26
29
  def process(hash, ttl, req_options)
27
30
  return if hash.empty?
28
31
 
32
+ @value_error = nil
29
33
  @ring.lock do
30
34
  servers = setup_requests(hash, ttl, req_options)
31
35
  finish_requests(servers)
32
36
  end
37
+ raise @value_error if @value_error
33
38
  rescue Dalli::RetryableNetworkError => e
34
39
  Dalli.logger.debug { e.inspect }
35
40
  Dalli.logger.debug { 'retrying pipelined sets because of network error' }
@@ -62,6 +67,10 @@ module Dalli
62
67
  server.request(:pipelined_set, key, value, ttl, req_options)
63
68
  rescue Dalli::NetworkError
64
69
  raise
70
+ rescue Dalli::MarshalError, Dalli::ValueOverMaxSize => e
71
+ # Nothing was written for this key, so the server's other keys are
72
+ # unaffected; raised by #process once the batch is complete
73
+ @value_error ||= e
65
74
  rescue DalliError => e
66
75
  Dalli.logger.debug { e.inspect }
67
76
  Dalli.logger.debug { "unable to set key #{key} for server #{server.name}" }
@@ -43,6 +43,7 @@ module Dalli
43
43
  @options = client_options.merge(user_creds)
44
44
  @raw_mode = client_options[:raw]
45
45
  @defer_drain = client_options[:defer_drain] ? true : false
46
+ @verify_reply_keys = client_options[:verify_reply_keys] ? true : false
46
47
  @value_marshaller = @raw_mode ? StringMarshaller.new(@options) : ValueMarshaller.new(@options)
47
48
  @connection_manager = ConnectionManager.new(hostname, port, socket_type, @options)
48
49
  end
@@ -79,8 +80,14 @@ module Dalli
79
80
 
80
81
  request_completed = true
81
82
  response
82
- rescue Dalli::MarshalError => e
83
- log_marshal_err(args.first, e)
83
+ rescue Dalli::MarshalError, Dalli::ValueOverMaxSize => e
84
+ log_marshal_err(args.first, e) if e.is_a?(Dalli::MarshalError)
85
+ # Raised only while the connection is clean: a value is marshalled
86
+ # before any of its request is written, and write_multi_req raises
87
+ # only after reading every reply. Closing the connection would
88
+ # discard earlier quiet writes still waiting in the write buffer.
89
+ @connection_manager.abort_request!
90
+ request_completed = true
84
91
  raise
85
92
  rescue Dalli::DalliError
86
93
  raise
@@ -153,6 +160,8 @@ module Dalli
153
160
  # avoiding intermediate Hash allocation. Returns nil.
154
161
  # Without a block, returns a Hash of { key => [value, cas] }.
155
162
  def pipeline_next_responses(&block)
163
+ return abandon_pipeline if @pipeline_out_of_step
164
+
156
165
  reconnect_on_pipeline_complete!
157
166
  values = nil
158
167
 
@@ -174,6 +183,13 @@ module Dalli
174
183
  # to the noop at the end of the pipeline
175
184
  finish_pipeline && break if status && key.nil?
176
185
 
186
+ # A key that wasn't requested means the replies are out of step
187
+ # (see expect_reply_keys): keep what was read so far, drop the rest
188
+ if unexpected_reply_key?(key)
189
+ abandon_pipeline
190
+ break
191
+ end
192
+
177
193
  # If the status is ok and the key is not nil, then this is a
178
194
  # getkq response with a value that we want to set in the response hash
179
195
  unless key.nil?
@@ -346,6 +362,7 @@ module Dalli
346
362
  # the expected state before pipeline_response_setup is called.
347
363
  response_buffer.clear
348
364
  response_buffer.raw = raw_request?(options)
365
+ expect_reply_keys(keys)
349
366
 
350
367
  # The terminating noop is sent by pipeline_response_setup
351
368
  write(quiet_get_requests(keys, options, return_cas: return_cas))
@@ -358,6 +375,7 @@ module Dalli
358
375
  # Initialize the response buffer for draining during send phase
359
376
  response_buffer.ensure_ready
360
377
  response_buffer.raw = raw_request?(options)
378
+ expect_reply_keys(keys)
361
379
 
362
380
  keys.each_slice(chunk_size) do |chunk|
363
381
  # Build and write this chunk of requests
@@ -390,6 +408,12 @@ module Dalli
390
408
  status, cas, key, value = response_buffer.process_single_getk_response
391
409
  break if status.nil? # No complete response available
392
410
 
411
+ # Handled by the next pipeline_next_responses, once sending is done
412
+ if unexpected_reply_key?(key)
413
+ @pipeline_out_of_step = true
414
+ break
415
+ end
416
+
393
417
  results[key] = [value, cas] unless key.nil?
394
418
  end
395
419
  rescue SystemCallError, Dalli::NetworkError
@@ -401,6 +425,42 @@ module Dalli
401
425
  @response_buffer ||= ResponseBuffer.new(@connection_manager, response_processor)
402
426
  end
403
427
 
428
+ # A multi-get only accepts replies for the keys it asked for. memcached
429
+ # names an item created by auto-create (ma or mg with N) for a
430
+ # base64-encoded key by its raw bytes, without the b flag; such a key
431
+ # can contain spaces or CR/LF, which split the reply header and make the
432
+ # replies after it parse as different ones. A key that wasn't requested
433
+ # is how that shows up.
434
+ # Only with the verify_reply_keys option, since building the set costs a
435
+ # multi-get about 15%. Dalli itself no longer creates such items (see
436
+ # Meta#decr_incr); the check guards against ones that older versions or
437
+ # other clients created, on memcached versions with the behavior.
438
+ def expect_reply_keys(keys)
439
+ @pipeline_out_of_step = false
440
+ @expected_reply_keys = (keys.to_set { |k| reply_key_form(k) } if @verify_reply_keys)
441
+ end
442
+
443
+ def unexpected_reply_key?(key)
444
+ !key.nil? && @expected_reply_keys && !@expected_reply_keys.include?(reply_key_form(key))
445
+ end
446
+
447
+ # Keys compared as bytes; ASCII keys compare equal in any encoding as is
448
+ def reply_key_form(key)
449
+ key.ascii_only? ? key : key.b
450
+ end
451
+
452
+ # Keeps what was read, drops the rest of this server's replies, and
453
+ # closes the connection so nothing unread is left on it. Not a socket
454
+ # failure, so it doesn't count toward socket_max_failures; the keys not
455
+ # read are simply missing from the result.
456
+ def abandon_pipeline
457
+ Dalli.logger.warn { "#{name} replied to a get_multi with a key that wasn't requested; closing the connection" }
458
+ @pipeline_out_of_step = false
459
+ response_buffer.clear
460
+ @connection_manager.close
461
+ {}
462
+ end
463
+
404
464
  # Called after the noop response is received at the end of a set
405
465
  # of pipelined gets
406
466
  def finish_pipeline
@@ -414,7 +474,10 @@ module Dalli
414
474
  @connection_manager.reconnect! 'pipelined get has completed' if pipeline_complete?
415
475
  end
416
476
 
477
+ # The single-server set_multi path passes its whole pairs hash as the
478
+ # first argument; log only its keys, never the values
417
479
  def log_marshal_err(key, err)
480
+ key = key.keys.join("', '") if key.is_a?(Hash)
418
481
  Dalli.logger.error "Marshalling error for key '#{key}': #{err.message}"
419
482
  Dalli.logger.error 'You are trying to cache a Ruby object which cannot be serialized to memcached.'
420
483
  end
@@ -60,8 +60,8 @@ module Dalli
60
60
  @pid = Process.pid
61
61
  @request_in_progress = false
62
62
  @quiet_responses_pending = false
63
- rescue SystemCallError, *TIMEOUT_ERRORS, EOFError, SocketError => e
64
- # SocketError = DNS resolution failure
63
+ rescue SystemCallError, *TIMEOUT_ERRORS, *SSL_ERRORS, EOFError, SocketError => e
64
+ # SocketError = DNS resolution failure; SSL_ERRORS = failed TLS handshake
65
65
  error_on_request!(e)
66
66
  end
67
67
 
@@ -188,10 +188,22 @@ module Dalli
188
188
  @request_in_progress = false
189
189
  end
190
190
 
191
+ # The longest reply line Dalli reads, in bytes, CRLF included. A meta
192
+ # reply header is at most about 500 bytes: "VA <size>" plus its flags,
193
+ # the largest being a 250-byte key base64-encoded to 336 bytes, a
194
+ # 20-digit CAS, a 32-byte opaque and a few numeric fields. The longest
195
+ # stats line (stats settings) carries a file path such as ext_path, up
196
+ # to PATH_MAX (4096). 8 KiB leaves room for both, while a server that
197
+ # never sends CRLF can't grow the read buffer without bound.
198
+ MAX_LINE_BYTES = 8 * 1024
199
+
191
200
  def read_line
192
201
  flush_write_buffer
193
- data = @sock.gets("\r\n")
202
+ data = @sock.gets("\r\n", MAX_LINE_BYTES)
194
203
  error_on_request!('EOF in read_line') if data.nil?
204
+ # Without its CRLF the line either ran past MAX_LINE_BYTES or was cut
205
+ # short by EOF; either way the connection is out of step
206
+ error_on_request!("Reply line without CRLF in the first #{MAX_LINE_BYTES} bytes") unless data.end_with?("\r\n")
195
207
  data
196
208
  rescue SystemCallError, *TIMEOUT_ERRORS, *SSL_ERRORS, EOFError => e
197
209
  error_on_request!(e)
@@ -234,12 +234,45 @@ module Dalli
234
234
  # rubocop:disable-next Metrics/ParameterLists
235
235
  def decr_incr(incr, key, delta, ttl, initial, options = nil)
236
236
  ttl = initial ? TtlSanitizer.sanitize(ttl) : nil # Only set a TTL if we want to set a value on miss
237
+ # memcached stores an item created by ma's auto-create (N/J) for a
238
+ # base64-encoded key without its binary-key flag, and later replies
239
+ # name it with the raw key bytes. Create such counters with an add.
240
+ return decr_incr_with_add(incr, key, delta, ttl, initial, options) if initial && KeyRegularizer.required?(key)
241
+
237
242
  write(RequestFormatter.meta_arithmetic(key: key, delta: delta, initial: initial, incr: incr, ttl: ttl,
238
243
  quiet: quiet?, **routing_token_kwargs(options)))
239
244
  finish_write(quiet?)
240
245
  response_processor.decr_incr unless quiet?
241
246
  end
242
247
 
248
+ # Like decr_incr with an initial value, without memcached's auto-create:
249
+ # on a miss the counter is added with the initial value (which is
250
+ # returned, as auto-create would), and the add losing a race to another
251
+ # client means the counter now exists, so the incr is tried again. In a
252
+ # quiet block both requests are sent and whichever applies takes effect.
253
+ # rubocop:disable-next Metrics/ParameterLists
254
+ def decr_incr_with_add(incr, key, delta, ttl, initial, options)
255
+ add_options = (options || {}).merge(raw: true) # counters are stored raw
256
+ return quiet_decr_incr_with_add(incr, key, delta, ttl, initial, add_options) if quiet?
257
+
258
+ 2.times do
259
+ value = decr_incr(incr, key, delta, nil, nil, options)
260
+ return value unless value.nil?
261
+
262
+ write_storage_req(:add, key, initial.to_s, ttl, nil, add_options, quiet: false)
263
+ return initial if response_processor.meta_set_with_cas
264
+ end
265
+ decr_incr(incr, key, delta, nil, nil, options)
266
+ end
267
+
268
+ # rubocop:disable-next Metrics/ParameterLists
269
+ def quiet_decr_incr_with_add(incr, key, delta, ttl, initial, add_options)
270
+ write(RequestFormatter.meta_arithmetic(key: key, delta: delta, initial: nil, incr: incr, ttl: nil,
271
+ quiet: true, **routing_token_kwargs(add_options)))
272
+ write_storage_req(:add, key, initial.to_s, ttl, nil, add_options, quiet: true)
273
+ finish_write(true)
274
+ end
275
+
243
276
  # Other Commands
244
277
  def flush(delay = 0)
245
278
  write(RequestFormatter.flush(delay: delay))
@@ -293,7 +326,7 @@ module Dalli
293
326
  buffer = RequestFormatter.multi_meta_get(keys, skip_flags: is_raw, **routing_token_kwargs(options))
294
327
  flushed_write(buffer)
295
328
  buffer.clear
296
- read_multi_get_responses(is_raw)
329
+ read_multi_get_responses(is_raw, keys)
297
330
  end
298
331
 
299
332
  # Stale-aware bulk get. Returns { key => { value:, cas:, stale:, miss: } }
@@ -310,12 +343,13 @@ module Dalli
310
343
  **routing_token_kwargs(options))
311
344
  flushed_write(buffer)
312
345
  buffer.clear
313
- read_multi_metadata_responses(is_raw)
346
+ read_multi_metadata_responses(is_raw, keys)
314
347
  end
315
348
 
316
349
  # Unlike read_multi_get_responses this locates tokens by flag rather than
317
350
  # by position, because the c flag shifts the key's index.
318
- def read_multi_metadata_responses(is_raw)
351
+ def read_multi_metadata_responses(is_raw, keys)
352
+ expect_reply_keys(keys)
319
353
  hash = {}
320
354
  while (line = @connection_manager.read_line)
321
355
  break if line.start_with?('MN')
@@ -331,23 +365,34 @@ module Dalli
331
365
  key = response_processor.key_from_tokens(tokens)
332
366
  next if key.nil?
333
367
 
368
+ raise_unexpected_reply_key(key) if unexpected_reply_key?(key)
369
+
334
370
  hash[key] = { value: @value_marshaller.retrieve(value, bitflags), cas: cas, stale: stale, miss: false }
335
371
  end
336
372
  hash
337
373
  end
338
374
 
339
- def read_multi_get_responses(is_raw)
375
+ def read_multi_get_responses(is_raw, keys)
376
+ expect_reply_keys(keys)
340
377
  hash = {}
341
378
  while (line = @connection_manager.read_line)
342
379
  break if line.start_with?('MN')
343
380
  next unless line.start_with?('VA ')
344
381
 
345
382
  key, value = parse_multi_get_value(line, is_raw)
383
+ raise_unexpected_reply_key(key) if unexpected_reply_key?(key)
346
384
  hash[key] = value if key
347
385
  end
348
386
  hash
349
387
  end
350
388
 
389
+ # The replies after a key that wasn't requested can't be trusted (see
390
+ # Base#expect_reply_keys). Raising ends the request, which closes the
391
+ # connection; the caller treats this server's keys as missing.
392
+ def raise_unexpected_reply_key(key)
393
+ raise Dalli::UnexpectedReplyKeyError, "#{name} replied with a key that wasn't requested: #{key.inspect}"
394
+ end
395
+
351
396
  # Reads the "VA <size> [f<flags>] k<key> ..." line in place rather than
352
397
  # splitting it into tokens
353
398
  def parse_multi_get_value(line, is_raw)
@@ -364,16 +409,27 @@ module Dalli
364
409
 
365
410
  # Single-server fast path for set_multi. Inlines request formatting to
366
411
  # minimize per-key overhead. Avoids PipelinedSetter server grouping.
412
+ #
413
+ # A value that can't be marshalled or is over value_max_bytes is skipped;
414
+ # the other keys are still stored, then the first such error is raised
415
+ # (as PipelinedSetter does on the multi-server path).
367
416
  def write_multi_req(pairs, ttl, req_options)
368
417
  ttl = TtlSanitizer.sanitize(ttl) if ttl
369
- entries = pairs.map do |key, raw_value|
418
+ value_error = nil
419
+ entries = pairs.filter_map do |key, raw_value|
370
420
  [key, @value_marshaller.store(key, raw_value, req_options)]
421
+ rescue Dalli::MarshalError, Dalli::ValueOverMaxSize => e
422
+ value_error ||= e
423
+ nil
371
424
  end
372
425
 
373
- buffer = RequestFormatter.multi_meta_set(entries, ttl: ttl, **routing_token_kwargs(req_options))
374
- flushed_write(buffer)
375
- buffer.clear
376
- response_processor.consume_all_responses_until_mn
426
+ unless entries.empty?
427
+ buffer = RequestFormatter.multi_meta_set(entries, ttl: ttl, **routing_token_kwargs(req_options))
428
+ flushed_write(buffer)
429
+ buffer.clear
430
+ response_processor.consume_all_responses_until_mn
431
+ end
432
+ raise value_error if value_error
377
433
  end
378
434
 
379
435
  # Single-server fast path for delete_multi. Writes all quiet delete requests
@@ -33,7 +33,7 @@ module Dalli
33
33
  username: uri.user,
34
34
  password: uri.password
35
35
  }
36
- [uri.host, normalize_port(uri.port), :tcp, DEFAULT_WEIGHT, auth_details]
36
+ [uri.hostname, normalize_port(uri.port), :tcp, DEFAULT_WEIGHT, auth_details]
37
37
  end
38
38
 
39
39
  def self.parse_non_uri(str)
@@ -61,11 +61,11 @@ module Dalli
61
61
  # in case of unix socket, allow only setting of weight, not port
62
62
  raise Dalli::DalliError, "Could not parse hostname #{res[0]}" if res[4]
63
63
 
64
- [nil, normalize_weight(res[3])]
64
+ [nil, normalize_weight(res[3], res[0])]
65
65
  end
66
66
 
67
67
  def self.attributes_for_tcp_socket(res)
68
- [normalize_port(res[3]), normalize_weight(res[4])]
68
+ [normalize_port(res[3]), normalize_weight(res[4], res[0])]
69
69
  end
70
70
 
71
71
  def self.normalize_host_from_match(str, res)
@@ -78,8 +78,13 @@ module Dalli
78
78
  Integer(port || DEFAULT_PORT)
79
79
  end
80
80
 
81
- def self.normalize_weight(weight)
82
- Integer(weight || DEFAULT_WEIGHT)
81
+ # A weight below 1 gives the server no points on the ring, and if every
82
+ # server has one the ring divides by a zero total weight.
83
+ def self.normalize_weight(weight, str)
84
+ value = Integer(weight || DEFAULT_WEIGHT)
85
+ raise ArgumentError, "Server weight must be at least 1, got #{value} in #{str}" if value < 1
86
+
87
+ value
83
88
  end
84
89
  end
85
90
  end
@@ -10,7 +10,8 @@ module Dalli
10
10
  ##
11
11
  class StringMarshaller
12
12
  DEFAULTS = {
13
- # max size of value in bytes (default is 1 MB, can be overriden with "memcached -I <size>")
13
+ # max memcached item size in bytes: value + key + ITEM_OVERHEAD_BYTES. Set it to
14
+ # memcached's -I value (default 1 MB).
14
15
  value_max_bytes: 1024 * 1024
15
16
  }.freeze
16
17
 
@@ -55,10 +56,7 @@ module Dalli
55
56
  private
56
57
 
57
58
  def error_if_over_max_value_bytes(key, value)
58
- return if value.bytesize <= value_max_bytes
59
-
60
- message = "Value for #{key} over max size: #{value_max_bytes} <= #{value.bytesize}"
61
- raise Dalli::ValueOverMaxSize, message
59
+ ValueMarshaller.error_if_over_max_value_bytes(key, value, value_max_bytes)
62
60
  end
63
61
  end
64
62
  end
@@ -84,11 +84,13 @@ module Dalli
84
84
  end
85
85
 
86
86
  # Checks whether we should apply compression when serializing a value
87
- # based on the specified options. Returns false unless the value
88
- # is greater than the minimum compression size. Otherwise returns
89
- # based on a method-level option if specified, falling back to the
90
- # server default.
87
+ # based on the specified options. Returns false for a raw request
88
+ # (raw reads ignore bitflags, so the value must be stored as given) or
89
+ # unless the value is greater than the minimum compression size.
90
+ # Otherwise returns based on a method-level option if specified,
91
+ # falling back to the server default.
91
92
  def compress_value?(value, req_options)
93
+ return false if req_options&.dig(:raw)
92
94
  return false unless value.bytesize >= compression_min_size
93
95
  return compress_by_default? unless req_options && !req_options[:compress].nil?
94
96
 
@@ -13,12 +13,30 @@ module Dalli
13
13
  extend Forwardable
14
14
 
15
15
  DEFAULTS = {
16
- # max size of value in bytes (default is 1 MB, can be overriden with "memcached -I <size>")
16
+ # max memcached item size in bytes: value + key + ITEM_OVERHEAD_BYTES. Set it to
17
+ # memcached's -I value (default 1 MB).
17
18
  value_max_bytes: 1024 * 1024
18
19
  }.freeze
19
20
 
20
21
  OPTIONS = DEFAULTS.keys.freeze
21
22
 
23
+ # memcached's -I limit applies to the whole item, not just the value. On
24
+ # 64-bit memcached 1.6 an item takes the value, the key, and 63 bytes of
25
+ # overhead: a 48-byte header, 8 bytes of CAS, 4 bytes of client flags,
26
+ # the key's NUL terminator and the value's trailing CRLF. (Measured
27
+ # against memcached 1.6.45 with -I 512k, 1m, 2m and 5m; an item stored
28
+ # with zero flags or with CAS disabled needs a few bytes less.)
29
+ ITEM_OVERHEAD_BYTES = 63
30
+
31
+ def self.error_if_over_max_value_bytes(key, value, value_max_bytes)
32
+ item_bytes = value.bytesize + key.bytesize + ITEM_OVERHEAD_BYTES
33
+ return if item_bytes <= value_max_bytes
34
+
35
+ message = "Value for #{key} over max size: #{value_max_bytes} <= #{item_bytes} " \
36
+ "(#{value.bytesize} value bytes + #{key.bytesize} key bytes + #{ITEM_OVERHEAD_BYTES} item overhead)"
37
+ raise Dalli::ValueOverMaxSize, message
38
+ end
39
+
22
40
  def_delegators :@value_serializer, :serializer
23
41
  def_delegators :@value_compressor, :compressor, :compression_min_size, :compress_by_default?
24
42
 
@@ -28,6 +46,7 @@ module Dalli
28
46
 
29
47
  @marshal_options =
30
48
  DEFAULTS.merge(client_options.slice(*OPTIONS))
49
+ @marshal_options[:value_max_bytes] = @marshal_options[:value_max_bytes].to_i
31
50
  end
32
51
 
33
52
  def store(key, value, options = nil)
@@ -49,10 +68,7 @@ module Dalli
49
68
  end
50
69
 
51
70
  def error_if_over_max_value_bytes(key, value)
52
- return if value.bytesize <= value_max_bytes
53
-
54
- message = "Value for #{key} over max size: #{value_max_bytes} <= #{value.bytesize}"
55
- raise Dalli::ValueOverMaxSize, message
71
+ self.class.error_if_over_max_value_bytes(key, value, value_max_bytes)
56
72
  end
57
73
  end
58
74
  end
@@ -16,9 +16,9 @@ module Dalli
16
16
  [Timeout::Error]
17
17
  end
18
18
 
19
- # SSL errors that occur during read/write operations (not during initial
20
- # handshake) should trigger reconnection. These indicate transient network
21
- # issues, not configuration problems.
19
+ # SSL errors during the handshake or during read/write operations count as
20
+ # a failed request: they trigger reconnection and, after
21
+ # socket_max_failures, mark the server down.
22
22
  SSL_ERRORS =
23
23
  if defined?(OpenSSL::SSL::SSLError)
24
24
  [OpenSSL::SSL::SSLError]
data/lib/dalli/socket.rb CHANGED
@@ -8,6 +8,12 @@ module Dalli
8
8
  # Various socket implementations used by Dalli.
9
9
  ##
10
10
  module Socket
11
+ # Timeout for establishing a connection: connect_timeout if set,
12
+ # otherwise socket_timeout.
13
+ def self.connect_timeout_for(options)
14
+ options[:connect_timeout] || options[:socket_timeout]
15
+ end
16
+
11
17
  ##
12
18
  # Common methods for all socket implementations.
13
19
  ##
@@ -162,11 +168,13 @@ module Dalli
162
168
  end
163
169
 
164
170
  def self.create_socket_with_timeout(host, port, options)
171
+ timeout = Dalli::Socket.connect_timeout_for(options)
165
172
  if supports_connect_timeout?
166
- sock = new(host, port, connect_timeout: options[:socket_timeout])
173
+ # resolv_timeout bounds the DNS lookup, which connect_timeout doesn't cover
174
+ sock = new(host, port, connect_timeout: timeout, resolv_timeout: timeout)
167
175
  yield(sock)
168
176
  else
169
- Timeout.timeout(options[:socket_timeout]) do
177
+ Timeout.timeout(timeout) do
170
178
  sock = new(host, port)
171
179
  yield(sock)
172
180
  end
@@ -228,12 +236,41 @@ module Dalli
228
236
  [seconds, microseconds].pack(timeval_pack_format(sock))
229
237
  end
230
238
 
239
+ # Closes the TCP socket if the handshake fails or times out, so a failed
240
+ # attempt doesn't hold a file descriptor until GC.
231
241
  def self.wrapping_ssl_socket(tcp_socket, host, ssl_context)
242
+ warn_if_unverified(ssl_context)
243
+ connected = false
232
244
  ssl_socket = Dalli::Socket::SSLSocket.new(tcp_socket, ssl_context)
233
245
  ssl_socket.hostname = host
234
246
  ssl_socket.sync_close = true
235
247
  ssl_socket.connect
248
+ connected = true
236
249
  ssl_socket
250
+ ensure
251
+ tcp_socket.close unless connected
252
+ end
253
+
254
+ # Class variable to track whether the TLS verification warning has been logged
255
+ @@ssl_verification_warning_logged = false # rubocop:disable Style/ClassVars
256
+
257
+ # Verification comes entirely from the caller's ssl_context, and a bare
258
+ # SSLContext.new verifies nothing, so warn (once per process) when the
259
+ # certificate or the hostname won't be checked.
260
+ def self.warn_if_unverified(ssl_context)
261
+ return if @@ssl_verification_warning_logged
262
+
263
+ problem = if !ssl_context.verify_mode.to_i.anybits?(::OpenSSL::SSL::VERIFY_PEER)
264
+ 'verify_mode is VERIFY_NONE, so the server certificate is not checked'
265
+ elsif !ssl_context.verify_hostname
266
+ 'verify_hostname is false, so the certificate is not checked against the hostname'
267
+ end
268
+ return unless problem
269
+
270
+ Dalli.logger.warn "SECURITY WARNING: Dalli's ssl_context #{problem}. " \
271
+ 'Use ssl_context.set_params(verify_mode: OpenSSL::SSL::VERIFY_PEER, ' \
272
+ 'verify_hostname: true) to verify the server.'
273
+ @@ssl_verification_warning_logged = true # rubocop:disable Style/ClassVars
237
274
  end
238
275
  end
239
276
 
@@ -261,7 +298,7 @@ module Dalli
261
298
  attr_accessor :options
262
299
 
263
300
  def self.open(path, options = {})
264
- Timeout.timeout(options[:socket_timeout]) do
301
+ Timeout.timeout(Dalli::Socket.connect_timeout_for(options)) do
265
302
  sock = new(path)
266
303
  sock.options = { path: path }.merge(options)
267
304
  init_socket_options(sock, options)
data/lib/dalli/version.rb CHANGED
@@ -1,7 +1,7 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Dalli
4
- VERSION = '5.2.2'
4
+ VERSION = '5.2.3'
5
5
 
6
6
  MIN_SUPPORTED_MEMCACHED_VERSION = '1.6.27'
7
7
  end
data/lib/dalli.rb CHANGED
@@ -31,6 +31,10 @@ module Dalli
31
31
  # socket/server communication error that can be retried
32
32
  class RetryableNetworkError < NetworkError; end
33
33
 
34
+ # a multi-get reply named a key that wasn't requested, so the rest of that
35
+ # server's replies can't be trusted
36
+ class UnexpectedReplyKeyError < DalliError; end
37
+
34
38
  # Implements the NullObject pattern to store an application-defined value for 'Key not found' responses.
35
39
  class NilObject; end # rubocop:disable Lint/EmptyClass
36
40
  NOT_FOUND = NilObject.new
@@ -2,7 +2,6 @@
2
2
 
3
3
  require 'rack/session/abstract/id'
4
4
  require 'dalli'
5
- require 'connection_pool'
6
5
  require 'English'
7
6
 
8
7
  module Rack
@@ -21,6 +20,11 @@ module Rack
21
20
  namespace: 'rack:session'
22
21
  }
23
22
 
23
+ # Our pool options and the ConnectionPool options they set. These are
24
+ # not passed on to Dalli::Client.
25
+ POOL_OPTIONS = { pool_size: :size, pool_timeout: :timeout }.freeze
26
+ private_constant :POOL_OPTIONS
27
+
24
28
  # Brings in a new Rack::Session::Dalli middleware with the given
25
29
  # `:memcache_server`. The server is either a hostname, or a
26
30
  # host-with-port string in the form of "host_name:port", or an array of
@@ -73,10 +77,18 @@ module Rack
73
77
  @data = build_data_source(options)
74
78
  end
75
79
 
76
- def call(*_args)
77
- super
80
+ # If the session is deleted from memcached while a request is running
81
+ # (for example, by a logout in another request), the session is not
82
+ # written back, so a logged-out session can't come back. The app's
83
+ # response is replaced with a 401 that expires the session cookie.
84
+ # The app has already run by then, so any side effects it had (database
85
+ # writes, emails, and so on) have happened.
86
+ def context(env, app = @app)
87
+ body = nil
88
+ super(env, ->(app_env) { app.call(app_env).tap { |response| body = response[2] } })
78
89
  rescue MissingSessionError
79
- [401, {}, ['Wrong session ID']]
90
+ body.close if body.respond_to?(:close)
91
+ missing_session_response(env)
80
92
  end
81
93
 
82
94
  def find_session(req, sid)
@@ -111,13 +123,23 @@ module Rack
111
123
  def delete_session(_req, sid, options)
112
124
  with_dalli_client do |dc|
113
125
  key = memcached_key_from_sid(sid)
114
- dc.delete(key) if key
126
+ delete_session_key(dc, key) if key
115
127
  generate_sid_with(dc) unless options[:drop]
116
128
  end
117
129
  end
118
130
 
119
131
  private
120
132
 
133
+ # A failed delete leaves the old session readable in memcached (e.g.
134
+ # after logout), so always log it, even though with_dalli_client then
135
+ # handles the error as before.
136
+ def delete_session_key(client, key)
137
+ client.delete(key)
138
+ rescue ::Dalli::DalliError, Errno::ECONNREFUSED => e
139
+ ::Dalli.logger.warn("#{self.class} failed to delete a session from memcached: #{e.class}: #{e.message}")
140
+ raise
141
+ end
142
+
121
143
  def memcached_key_from_sid(sid)
122
144
  sid.private_id if sid.respond_to?(:private_id)
123
145
  end
@@ -162,6 +184,12 @@ module Rack
162
184
  end
163
185
  end
164
186
 
187
+ def missing_session_response(env)
188
+ options = env[Rack::RACK_SESSION_OPTIONS] || @default_options
189
+ cookie = Rack::Utils.delete_set_cookie_header(key, options.slice(:path, :domain))
190
+ [401, { 'content-type' => 'text/plain', 'set-cookie' => cookie }, ['Wrong session ID']]
191
+ end
192
+
165
193
  def write_session_safely!(dalli_client, sid, persisted_data, write_args:)
166
194
  if persisted_data && persisted_data[:id] == sid # That means that we update the existing session
167
195
  # Override the session only if it still exists in the store!
@@ -181,15 +209,14 @@ module Rack
181
209
  end
182
210
 
183
211
  def retrieve_client_options(options)
184
- # Filter out Rack::Session-specific options and apply our defaults
185
- filtered_opts = options.reject { |k, _| DEFAULT_OPTIONS.key? k }
212
+ # Filter out Rack::Session-specific and pool options and apply our defaults
213
+ filtered_opts = options.reject { |k, _| DEFAULT_OPTIONS.key?(k) || POOL_OPTIONS.key?(k) }
186
214
  DEFAULT_DALLI_OPTIONS.merge(filtered_opts)
187
215
  end
188
216
 
189
217
  def retrieve_pool_options(options)
190
- {}.tap do |pool_options|
191
- pool_options[:size] = options.delete(:pool_size) if options[:pool_size]
192
- pool_options[:timeout] = options.delete(:pool_timeout) if options[:pool_timeout]
218
+ POOL_OPTIONS.each_with_object({}) do |(option, pool_option), pool_options|
219
+ pool_options[pool_option] = options[option] if options[option]
193
220
  end
194
221
  end
195
222
 
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: dalli
3
3
  version: !ruby/object:Gem::Version
4
- version: 5.2.2
4
+ version: 5.2.3
5
5
  platform: ruby
6
6
  authors:
7
7
  - Peter M. Goldstein
@@ -89,7 +89,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
89
89
  - !ruby/object:Gem::Version
90
90
  version: '0'
91
91
  requirements: []
92
- rubygems_version: 4.0.21
92
+ rubygems_version: 4.0.23
93
93
  specification_version: 4
94
94
  summary: High performance memcached client for Ruby
95
95
  test_files: []