dalli 5.2.1 → 5.2.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 42431b1f305b3fe3d3923c2b66f3d1edfb402da378c434a587bb2c85d6688055
4
- data.tar.gz: 39bc473979fb49cd488853d39a64675884ce89951e96a7b5925dc88c8307219a
3
+ metadata.gz: cd910ad2e0c6e5aa47b8591e9ba8c5806b6208fbab0ae0852637aecc48939051
4
+ data.tar.gz: bd0db96211060011acb2135feb337d8acc88638b08c57b3fa875445695a09b59
5
5
  SHA512:
6
- metadata.gz: 810da4f09ea75bfa616471a48686cb0804b820b3a629c1ec53c3ecc7146c76d94e7afa729ce3361287ccafaa71d97a0ed5053f0b6adc0d8cea91c5de81070231
7
- data.tar.gz: c18a6a1f3d1dac3fcf5f640c8714ff68e70e07dc8d154d1471fc62c9ccb5ee6a970a3d079b042d4c5337ad11f39090e69022ef6eb34dac281906e9834863b285
6
+ metadata.gz: 82da7aebd50661e1ab9af25dab7d66e5f70cb15260ff9ca4b3476e63eeb60cbb2ad1299718e56312f9a6a61d7a2b14d394fcc470325329f1152db42ac921ee6d
7
+ data.tar.gz: a036421fac121aa342adc3beced711ce64a1bd241673a9314f1f1454a951515890732c4c9651e52baf65c94cd01c4b5cbb655c8dcca4fa4db03d656c3cfc4abe
data/CHANGELOG.md CHANGED
@@ -4,6 +4,35 @@ Dalli Changelog
4
4
  Unreleased
5
5
  ==========
6
6
 
7
+ 5.2.2
8
+ ==========
9
+
10
+ Security:
11
+
12
+ - Keep the caller's key when retrying a request (GHSA-m252-9cgf-vx2w)
13
+ - With a `namespace`, a request retried after a transient network error (a timeout, or a connection closed by memcached or a proxy) applied the namespace a second time, so a retried read could return a different key's value and a retried write could overwrite a different key
14
+ - Affects single-key operations since 5.0.3, single-server `get_multi` since 5.1.0, and `get_with_metadata` and `fetch_with_lock` since 4.1.0; fixed in 5.2.2, 5.1.4, 5.0.10 and 4.3.7. Clients without a namespace aren't affected
15
+ - Complete the fix for per-request `raw: true` on reads (GHSA-wr87-m4jw-29x5)
16
+ - A read made with `raw: true` doesn't ask for flags, but a reply carrying them anyway (from a proxy or a hostile server) still had its value deserialized. Raw reads now ignore flags in the reply
17
+ - Affects 5.2.1 and earlier; fixed in 5.2.2, 5.1.4, 5.0.10, 4.3.7 and 3.2.13
18
+
19
+ Bug fixes:
20
+
21
+ - `get_with_metadata` and `fetch_with_lock` retried the final error raised when a server is marked down, so with `down_retry_delay: 0` they retried an unresponsive server forever. They now retry only retryable errors, like other operations
22
+ - Fix a regression in 5.2.1: a `get_multi` that didn't finish within `socket_timeout` counted toward `socket_max_failures`, so two slow `get_multi` calls in a row marked a healthy server down. It now just closes the connection
23
+ - Fix a regression in 5.2.1: over TLS, every request buffered in a `quiet` block was sent as its own TLS record and system call (2.5 times slower for a block of 2000 deletes). Each flush is one write again
24
+ - Handle malformed replies from a broken or hostile server or proxy: a negative value size, a hit with no key, or a hit with no `s` flag could leave the connection out of step or raise a non-Dalli error
25
+ - With `decompressed_max_bytes` in effect, data after the end of a compressed value's stream was returned as part of the value. It's now ignored, as without the limit
26
+ - With a `digest_class` whose digests aren't short hex strings, shortening a long key could loop forever. It now raises `ArgumentError`
27
+
28
+ Notes:
29
+
30
+ - Document how long keys are shortened, and that a short key written in the same form names the same item, in the README's security note (#1200)
31
+
32
+ Development:
33
+
34
+ - Run the Tests, RuboCop and Profiles workflows on pushes to `main` and the `*-stable` branches only, so a pull request's branch isn't tested twice (once for the push and once for the pull request) (#1198)
35
+
7
36
  5.2.1
8
37
  ==========
9
38
 
data/README.md CHANGED
@@ -97,6 +97,19 @@ Dalli::Client.new('localhost:11211', serializer: Dalli::JSONSerializer)
97
97
 
98
98
  Per-request `raw: true` returns the stored bytes without deserializing or decompressing them, on every read method (`get`, `get_multi`, `get_cas`, `get_with_metadata` and `fetch`).
99
99
 
100
+ ### Long keys
101
+
102
+ memcached limits keys to 250 bytes. Dalli shortens a longer key (counting its namespace) to the start of the key, followed by `:md5:` and a hex digest of the whole key:
103
+
104
+ ```ruby
105
+ dc.set("report:#{'x' * 300}", 'data')
106
+ # stored under "report:xxx...xxx:md5:<32 hex characters>"
107
+ ```
108
+
109
+ The shortened key is an ordinary memcached key, so a short key written in exactly that form names the same item, and reading or writing it reads or overwrites the long key's value. This only matters if untrusted input can choose an entire cache key. If it can, hash the untrusted part yourself so the key never needs shortening, or reject keys that contain `:md5:`.
110
+
111
+ The digest is MD5 by default. `digest_class:` takes any object that responds to `hexdigest`, such as `Digest::SHA256`. Changing it renames every shortened key, so existing entries for long keys are missed once.
112
+
100
113
  See the [5.0-Upgrade.md](5.0-Upgrade.md) guide for upgrade information.
101
114
 
102
115
  ## OpenTelemetry Tracing
data/lib/dalli/client.rb CHANGED
@@ -144,14 +144,18 @@ module Dalli
144
144
  #
145
145
  def get_with_metadata(key, options = {})
146
146
  validate_routing_tokens!(options)
147
- key = key.to_s
148
- key = @key_manager.validate_key(key)
147
+ # A retry re-runs this method with its original arguments, so `key` must
148
+ # stay as the caller gave it; reassigning it would add the namespace again.
149
+ validated_key = @key_manager.validate_key(key.to_s)
149
150
 
150
- server = ring.server_for_key(key)
151
- Instrumentation.trace('get_with_metadata', trace_attrs('get_with_metadata', key, server)) do
152
- server.request(:meta_get, key, options)
151
+ server = ring.server_for_key(validated_key)
152
+ Instrumentation.trace('get_with_metadata', trace_attrs('get_with_metadata', validated_key, server)) do
153
+ server.request(:meta_get, validated_key, options)
153
154
  end
154
- rescue NetworkError => e
155
+ # Only a retryable error is retried. The NetworkError raised when a server
156
+ # is marked down is final, as in #perform; retrying it looped forever once
157
+ # down_retry_delay let the same server be tried again.
158
+ rescue RetryableNetworkError => e
155
159
  Dalli.logger.debug { e.inspect }
156
160
  Dalli.logger.debug { 'retrying get_with_metadata with new server' }
157
161
  retry
@@ -347,14 +351,17 @@ module Dalli
347
351
  validate_integer!(:lock_ttl, lock_ttl)
348
352
  validate_integer!(:recache_threshold, recache_threshold)
349
353
  validate_routing_tokens!(req_options)
350
- key = key.to_s
351
- key = @key_manager.validate_key(key)
354
+ # Left unchanged for the retry below; see get_with_metadata
355
+ validated_key = @key_manager.validate_key(key.to_s)
352
356
 
353
- server = ring.server_for_key(key)
354
- Instrumentation.trace('fetch_with_lock', trace_attrs('fetch_with_lock', key, server)) do
355
- fetch_with_lock_request(key, ttl, lock_ttl, recache_threshold, req_options, &block)
357
+ server = ring.server_for_key(validated_key)
358
+ Instrumentation.trace('fetch_with_lock', trace_attrs('fetch_with_lock', validated_key, server)) do
359
+ fetch_with_lock_request(validated_key, ttl, lock_ttl, recache_threshold, req_options, &block)
356
360
  end
357
- rescue NetworkError => e
361
+ # Only a retryable error is retried. The NetworkError raised when a server
362
+ # is marked down is final, as in #perform; retrying it looped forever once
363
+ # down_retry_delay let the same server be tried again.
364
+ rescue RetryableNetworkError => e
358
365
  Dalli.logger.debug { e.inspect }
359
366
  Dalli.logger.debug { 'retrying fetch_with_lock with new server' }
360
367
  retry
@@ -787,10 +794,11 @@ module Dalli
787
794
  # to route to at all is still silent, matching Ring#keys_grouped_by_server
788
795
  # dropping a key it can't route on both the single- and multi-server paths.
789
796
  def single_server_get_multi(keys, req_options = nil)
790
- keys.map! { |k| @key_manager.validate_key(k.to_s) }
797
+ # A new array, so the retry below starts again from the caller's keys
798
+ validated_keys = keys.map { |k| @key_manager.validate_key(k.to_s) }
791
799
  return {} unless (server = single_server)
792
800
 
793
- result = server.request(:read_multi_req, keys, req_options)
801
+ result = server.request(:read_multi_req, validated_keys, req_options)
794
802
  result.transform_keys! { |k| @key_manager.key_without_namespace(k) }
795
803
  result
796
804
  rescue Dalli::RetryableNetworkError => e
@@ -957,18 +965,20 @@ module Dalli
957
965
  # rubocop:enable Naming/MethodParameterName
958
966
  return yield if block_given?
959
967
 
960
- key = key.to_s
961
- key = @key_manager.validate_key(key)
968
+ # `retry` below re-runs this method with its original arguments, so `key`
969
+ # must stay as the caller gave it: reassigning it would apply the
970
+ # namespace a second time and send the retry to a different key.
971
+ validated_key = @key_manager.validate_key(key.to_s)
962
972
 
963
- server = ring.server_for_key(key)
973
+ server = ring.server_for_key(validated_key)
964
974
 
965
975
  if Instrumentation.enabled?
966
976
  op_name = op.name
967
- Instrumentation.trace(op_name, trace_attrs(op_name, key, server)) do
968
- server.request(op, key, ...)
977
+ Instrumentation.trace(op_name, trace_attrs(op_name, validated_key, server)) do
978
+ server.request(op, validated_key, ...)
969
979
  end
970
980
  else
971
- server.request(op, key, ...)
981
+ server.request(op, validated_key, ...)
972
982
  end
973
983
  rescue RetryableNetworkError => e
974
984
  Dalli.logger.debug { e.inspect }
@@ -39,7 +39,12 @@ module Dalli
39
39
  out = String.new(encoding: Encoding::BINARY)
40
40
  0.step(data.bytesize - 1, INFLATE_SLICE_BYTES) do |pos|
41
41
  inflate_slice(inflater, data.byteslice(pos, INFLATE_SLICE_BYTES), out, max_bytes)
42
+ # Anything after the end of the stream isn't part of the value, as
43
+ # with Zlib::Inflate.inflate; finish would return it as output
44
+ break if inflater.finished?
42
45
  end
46
+ return out if inflater.finished?
47
+
43
48
  append_within_limit(out, inflater.finish, max_bytes)
44
49
  ensure
45
50
  inflater.close
@@ -122,15 +122,18 @@ module Dalli
122
122
  ##
123
123
  def truncated_key(key)
124
124
  digest = digest_class.hexdigest(key)
125
- prefix = key[0, prefix_length(digest)]
125
+ prefix = key[0, [prefix_length(digest), 0].max]
126
126
  truncated = "#{prefix}#{TRUNCATED_KEY_SEPARATOR}#{digest}"
127
127
  # A prefix with whitespace or non-ASCII characters sends the whole key
128
128
  # base64-encoded, which can still be too long on the wire; shorten it.
129
- while wire_length(truncated) > MAX_KEY_LENGTH
129
+ while wire_length(truncated) > MAX_KEY_LENGTH && !prefix.empty?
130
130
  prefix = prefix[0...-1]
131
131
  truncated = "#{prefix}#{TRUNCATED_KEY_SEPARATOR}#{digest}"
132
132
  end
133
- truncated
133
+ return truncated if wire_length(truncated) <= MAX_KEY_LENGTH
134
+
135
+ # Only possible with a digest_class whose digests aren't short hex strings
136
+ raise ArgumentError, "#{digest_class} digests are too long to fit in a truncated key"
134
137
  end
135
138
 
136
139
  # memcached's key limit applies to the key as sent: in bytes, and after
@@ -203,10 +203,13 @@ module Dalli
203
203
  @connection_manager.abort_request!
204
204
  return true unless connected?
205
205
 
206
- # Closes the connection, which ensures that our connection
207
- # is in a clean state for future requests
208
- @connection_manager.error_on_request!('External timeout')
209
- rescue NetworkError
206
+ # Closes the connection, which ensures that our connection is in a
207
+ # clean state for future requests. The get_multi ran out of time
208
+ # overall rather than hitting a socket error, so it doesn't count
209
+ # toward socket_max_failures: two slow get_multis in a row would
210
+ # otherwise mark a healthy server down.
211
+ Dalli.logger.warn { "#{name} didn't finish a get_multi within socket_timeout; closing the connection" }
212
+ @connection_manager.close
210
213
  true
211
214
  end
212
215
 
@@ -342,6 +345,7 @@ module Dalli
342
345
  # Use clear (not reset) to keep pipeline_complete? = true, which is
343
346
  # the expected state before pipeline_response_setup is called.
344
347
  response_buffer.clear
348
+ response_buffer.raw = raw_request?(options)
345
349
 
346
350
  # The terminating noop is sent by pipeline_response_setup
347
351
  write(quiet_get_requests(keys, options, return_cas: return_cas))
@@ -353,6 +357,7 @@ module Dalli
353
357
  def pipelined_get_interleaved(keys, chunk_size, results, options = nil)
354
358
  # Initialize the response buffer for draining during send phase
355
359
  response_buffer.ensure_ready
360
+ response_buffer.raw = raw_request?(options)
356
361
 
357
362
  keys.each_slice(chunk_size) do |chunk|
358
363
  # Build and write this chunk of requests
@@ -352,13 +352,32 @@ module Dalli
352
352
 
353
353
  private
354
354
 
355
+ # One write per flush. A TLS socket sends each argument of
356
+ # write(*parts) as its own record and system call, so the parts are
357
+ # joined first.
355
358
  def flush_write_buffer
356
359
  return if @write_buffer.empty?
357
360
 
358
- @sock.write(*@write_buffer)
361
+ @sock.write(@write_buffer.size == 1 ? @write_buffer.first : joined_write_buffer)
359
362
  discard_write_buffer
360
363
  end
361
364
 
365
+ # Joined as bytes: requests can mix UTF-8 and binary strings, which
366
+ # String#<< refuses to combine when both hold non-ASCII bytes
367
+ if String.method_defined?(:append_as_bytes)
368
+ def joined_write_buffer
369
+ out = String.new(capacity: @write_buffer_bytes, encoding: Encoding::BINARY)
370
+ @write_buffer.each { |part| out.append_as_bytes(part) }
371
+ out
372
+ end
373
+ else
374
+ def joined_write_buffer
375
+ out = String.new(capacity: @write_buffer_bytes, encoding: Encoding::BINARY)
376
+ @write_buffer.each { |part| out << part.b }
377
+ out
378
+ end
379
+ end
380
+
362
381
  def discard_write_buffer
363
382
  @write_buffer.clear
364
383
  @write_buffer_bytes = 0
@@ -38,7 +38,7 @@ module Dalli
38
38
  RequestFormatter.plain_meta_get(key, skip_flags)
39
39
  end
40
40
  flushed_write(req)
41
- response_processor.meta_get_with_value(cache_nils: cache_nils?(options))
41
+ response_processor.meta_get_with_value(cache_nils: cache_nils?(options), raw: raw_request?(options))
42
42
  end
43
43
 
44
44
  def quiet_get_request(key, options = nil)
@@ -59,7 +59,7 @@ module Dalli
59
59
  skip_flags = raw_mode? || (options && options[:raw])
60
60
  req = RequestFormatter.meta_get(key: key, ttl: ttl, skip_flags: skip_flags, **routing_token_kwargs(options))
61
61
  flushed_write(req)
62
- response_processor.meta_get_with_value(cache_nils: cache_nils?(options))
62
+ response_processor.meta_get_with_value(cache_nils: cache_nils?(options), raw: raw_request?(options))
63
63
  end
64
64
 
65
65
  def touch(key, ttl)
@@ -75,7 +75,7 @@ module Dalli
75
75
  req = RequestFormatter.meta_get(key: key, value: true, return_cas: true, skip_flags: raw_request?(options),
76
76
  **routing_token_kwargs(options))
77
77
  flushed_write(req)
78
- response_processor.meta_get_with_value_and_cas
78
+ response_processor.meta_get_with_value_and_cas(raw: raw_request?(options))
79
79
  end
80
80
 
81
81
  # Comprehensive meta get with support for all metadata flags.
@@ -116,7 +116,7 @@ module Dalli
116
116
  response_processor.meta_get_with_metadata(
117
117
  cache_nils: cache_nils?(options), return_hit_status: options[:return_hit_status],
118
118
  return_last_access: options[:return_last_access],
119
- return_ttl_remaining: options[:return_ttl_remaining]
119
+ return_ttl_remaining: options[:return_ttl_remaining], raw: raw_request?(options)
120
120
  )
121
121
  end
122
122
 
@@ -322,7 +322,9 @@ module Dalli
322
322
  next unless line.start_with?('VA ')
323
323
 
324
324
  tokens = line.chomp!(TERMINATOR).split
325
- value = @connection_manager.read(tokens[1].to_i + TERMINATOR.bytesize)&.chomp!(TERMINATOR)
325
+ size = tokens[1].to_i
326
+ response_processor.check_value_size!(size)
327
+ value = @connection_manager.read(size + TERMINATOR.bytesize)&.chomp!(TERMINATOR)
326
328
  stale = response_processor.stale_from_tokens(tokens)
327
329
  cas = response_processor.cas_from_tokens(tokens)
328
330
  bitflags = is_raw ? 0 : response_processor.bitflags_from_tokens(tokens)
@@ -350,7 +352,9 @@ module Dalli
350
352
  # splitting it into tokens
351
353
  def parse_multi_get_value(line, is_raw)
352
354
  processor = response_processor
353
- value = @connection_manager.read(processor.size_from_va_line(line) + TERMINATOR.bytesize)&.chomp!(TERMINATOR)
355
+ size = processor.size_from_va_line(line)
356
+ processor.check_value_size!(size)
357
+ value = @connection_manager.read(size + TERMINATOR.bytesize)&.chomp!(TERMINATOR)
354
358
  key = processor.key_from_va_line(line)
355
359
  return unless key
356
360
 
@@ -15,6 +15,9 @@ module Dalli
15
15
  # when advancing through parsed responses.
16
16
  ##
17
17
  class ResponseBuffer
18
+ # Whether the current pipeline was requested with raw: true
19
+ attr_writer :raw
20
+
18
21
  def initialize(io_source, response_processor)
19
22
  @io_source = io_source
20
23
  @response_processor = response_processor
@@ -39,7 +42,7 @@ module Dalli
39
42
  # Attempts to process a single response from the buffer,
40
43
  # advancing the offset past the consumed bytes.
41
44
  def process_single_getk_response
42
- response = @response_processor.getk_response_from_buffer(@buffer, @offset)
45
+ response = @response_processor.getk_response_from_buffer(@buffer, @offset, raw: @raw || false)
43
46
  @offset += response.pop
44
47
  response
45
48
  end
@@ -51,12 +51,15 @@ module Dalli
51
51
  @value_marshaller = value_marshaller
52
52
  end
53
53
 
54
- def meta_get_with_value(cache_nils: false)
54
+ # raw: true (a request made with raw: true) returns the value as stored,
55
+ # ignoring any flags in the reply. The request didn't ask for flags, so
56
+ # a reply carrying them anyway must not get the value deserialized.
57
+ def meta_get_with_value(cache_nils: false, raw: false)
55
58
  line = read_line
56
59
  # A hit ("VA <size> f<flags>") is parsed in place rather than split
57
60
  # into tokens, which saves several allocations on the hottest path.
58
61
  if line&.start_with?(VA_PREFIX)
59
- return @value_marshaller.retrieve(read_data(size_from_va_line(line)), bitflags_from_va_line(line))
62
+ return retrieve(read_data(size_from_va_line(line)), raw ? 0 : bitflags_from_va_line(line), raw)
60
63
  end
61
64
 
62
65
  tokens = line&.split || []
@@ -64,7 +67,7 @@ module Dalli
64
67
  when EN
65
68
  cache_nils ? ::Dalli::NOT_FOUND : nil
66
69
  when VA # only reached for an unusually formatted hit line
67
- @value_marshaller.retrieve(read_data(tokens[1].to_i), bitflags_from_tokens(tokens))
70
+ retrieve(read_data(tokens[1].to_i), raw ? 0 : bitflags_from_tokens(tokens), raw)
68
71
  when HD
69
72
  true
70
73
  else
@@ -72,14 +75,14 @@ module Dalli
72
75
  end
73
76
  end
74
77
 
75
- def meta_get_with_value_and_cas
78
+ def meta_get_with_value_and_cas(raw: false)
76
79
  tokens = error_on_unexpected!(T_VA_EN_HD)
77
80
  return [nil, 0] if tokens.first == EN
78
81
 
79
82
  cas = cas_from_tokens(tokens)
80
83
  return [nil, cas] unless tokens.first == VA
81
84
 
82
- [@value_marshaller.retrieve(read_data(tokens[1].to_i), bitflags_from_tokens(tokens)), cas]
85
+ [retrieve(read_data(tokens[1].to_i), raw ? 0 : bitflags_from_tokens(tokens), raw), cas]
83
86
  end
84
87
 
85
88
  def meta_get_without_value
@@ -99,13 +102,13 @@ module Dalli
99
102
  # Used by meta_get for comprehensive metadata retrieval.
100
103
  # Supports thundering herd protection (N/R flags) and metadata flags (h/l/u).
101
104
  def meta_get_with_metadata(cache_nils: false, return_hit_status: false, return_last_access: false,
102
- return_ttl_remaining: false)
105
+ return_ttl_remaining: false, raw: false)
103
106
  tokens = error_on_unexpected!(T_VA_EN_HD)
104
107
  result = build_metadata_result(tokens)
105
108
  result[:hit_before] = hit_status_from_tokens(tokens) if return_hit_status
106
109
  result[:last_access] = last_access_from_tokens(tokens) if return_last_access
107
110
  result[:ttl_remaining] = ttl_remaining_from_tokens(tokens) if return_ttl_remaining
108
- result[:value] = parse_value_from_tokens(tokens, cache_nils)
111
+ result[:value] = parse_value_from_tokens(tokens, cache_nils, raw)
109
112
  result
110
113
  end
111
114
 
@@ -122,11 +125,11 @@ module Dalli
122
125
  }
123
126
  end
124
127
 
125
- def parse_value_from_tokens(tokens, cache_nils)
128
+ def parse_value_from_tokens(tokens, cache_nils, raw = false) # rubocop:disable Style/OptionalBooleanParameter
126
129
  return cache_nils ? ::Dalli::NOT_FOUND : nil if tokens.first == EN
127
130
  return unless tokens.first == VA
128
131
 
129
- @value_marshaller.retrieve(read_data(tokens[1].to_i), bitflags_from_tokens(tokens))
132
+ retrieve(read_data(tokens[1].to_i), raw ? 0 : bitflags_from_tokens(tokens), raw)
130
133
  end
131
134
 
132
135
  def meta_set_with_cas
@@ -229,13 +232,15 @@ module Dalli
229
232
  # The remaining three values in the array are the ResponseHeader,
230
233
  # key, and value.
231
234
  ##
232
- def getk_response_from_buffer(buf, offset = 0)
235
+ # raw: true returns values as stored, ignoring flags in the replies;
236
+ # see meta_get_with_value.
237
+ def getk_response_from_buffer(buf, offset = 0, raw: false)
233
238
  # Find the header terminator starting from offset
234
239
  term_idx = buf.byteindex(TERMINATOR, offset)
235
240
  return [0] unless term_idx
236
241
 
237
242
  if buf.byteslice(offset, VA_PREFIX.bytesize) == VA_PREFIX
238
- response = va_response_from_buffer(buf, offset, term_idx)
243
+ response = va_response_from_buffer(buf, offset, term_idx, raw)
239
244
  return response if response
240
245
  end
241
246
 
@@ -256,7 +261,9 @@ module Dalli
256
261
  when BYTE_B then base64 ||= token == 'b'
257
262
  end
258
263
  end
259
- body_len = flag_int(size, 's')
264
+ # memcached also gives the size positionally ("VA <size> ..."); use it
265
+ # when there's no s flag rather than reading the value as replies
266
+ body_len = size ? flag_int(size, 's') : positional_size(tokens)
260
267
 
261
268
  # A complete response with no body. Only the terminating noop's MN
262
269
  # ends the pipeline. Any other bodyless reply (CLIENT_ERROR,
@@ -276,8 +283,12 @@ module Dalli
276
283
  # The full response is in our buffer, so parse it and return
277
284
  # the values
278
285
  body = buf.byteslice(offset + header_len, body_len)
279
- value = @value_marshaller.retrieve(body, flag_int(bitflags, 'f'))
280
- key = key ? key.delete_prefix!('k') : 0
286
+ value = retrieve(body, raw ? 0 : flag_int(bitflags, 'f'), raw)
287
+ # A hit with no key can't be matched to a request: skip it (false
288
+ # status) rather than yield it, or take it for the end of the pipeline
289
+ return [false, resp_size] unless key
290
+
291
+ key = key.delete_prefix!('k')
281
292
  key = KeyRegularizer.decode(key) if base64
282
293
  [tokens.first == VA, flag_int(cas, 'c'), key, value, resp_size]
283
294
  end
@@ -286,7 +297,7 @@ module Dalli
286
297
  # without allocating the header, its tokens or the flag strings.
287
298
  # Gives the same result as the token path. Returns nil to fall back to
288
299
  # it when the header has no s flag or a zero size.
289
- def va_response_from_buffer(buf, offset, term_idx)
300
+ def va_response_from_buffer(buf, offset, term_idx, raw = false) # rubocop:disable Style/OptionalBooleanParameter
290
301
  size = bitflags = cas = key = nil
291
302
  base64 = false
292
303
  pos = buf.byteindex(' ', offset + VA_PREFIX.bytesize)
@@ -309,9 +320,17 @@ module Dalli
309
320
  resp_size = header_len + size + TERMINATOR.length
310
321
  return [0] unless buf.bytesize >= offset + resp_size
311
322
 
312
- value = @value_marshaller.retrieve(buf.byteslice(offset + header_len, size), bitflags || 0)
313
- key = KeyRegularizer.decode(key) if base64 && key
314
- [true, cas || 0, key || 0, value, resp_size]
323
+ value = retrieve(buf.byteslice(offset + header_len, size), raw ? 0 : bitflags || 0, raw)
324
+ # Skipped like the token path does; see getk_response_from_buffer
325
+ return [false, resp_size] unless key
326
+
327
+ key = KeyRegularizer.decode(key) if base64
328
+ [true, cas || 0, key, value, resp_size]
329
+ end
330
+
331
+ # Values from a raw request are returned as stored
332
+ def retrieve(value, bitflags, raw)
333
+ raw ? value : @value_marshaller.retrieve(value, bitflags)
315
334
  end
316
335
 
317
336
  # Integer value of a flag's token in buf, after its one-byte prefix,
@@ -431,6 +450,11 @@ module Dalli
431
450
 
432
451
  # A pipelined reply claiming an impossible size would otherwise have the
433
452
  # buffer wait for (and accumulate) that many bytes
453
+ # The size given right after "VA", or 0 when it isn't a number
454
+ def positional_size(tokens)
455
+ tokens.first == VA && tokens[1]&.match?(/\A\d+\z/) ? tokens[1].to_i : 0
456
+ end
457
+
434
458
  def check_value_size!(size)
435
459
  return if size.between?(0, MAX_VALUE_BYTES)
436
460
 
@@ -447,6 +471,9 @@ module Dalli
447
471
  end
448
472
 
449
473
  def read_data(data_size)
474
+ # Checked before the terminator is added: -1 or -2 would otherwise
475
+ # read 1 or 0 bytes and leave the connection out of step
476
+ check_value_size!(data_size)
450
477
  @io_source.read(data_size + TERMINATOR.bytesize)&.chomp!(TERMINATOR)
451
478
  end
452
479
  end
data/lib/dalli/version.rb CHANGED
@@ -1,7 +1,7 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Dalli
4
- VERSION = '5.2.1'
4
+ VERSION = '5.2.2'
5
5
 
6
6
  MIN_SUPPORTED_MEMCACHED_VERSION = '1.6.27'
7
7
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: dalli
3
3
  version: !ruby/object:Gem::Version
4
- version: 5.2.1
4
+ version: 5.2.2
5
5
  platform: ruby
6
6
  authors:
7
7
  - Peter M. Goldstein