dalli 4.3.2 → 5.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 7eda4c85eb715e06ef3d8b34ea8e33a3cde89f19379a758f105363457451a034
4
- data.tar.gz: 36ae8b603f8f778edbbf3bed9b7504ff1a440382bf5520423f49c39961f04a55
3
+ metadata.gz: 16d7d5950137f4a51ae41392eaa576732595ac4020ae7be972d6fb4b33b9861d
4
+ data.tar.gz: ebb799522259a9a32a86e1dbfd13e810427f787f8110415509288655a5a5460b
5
5
  SHA512:
6
- metadata.gz: b9a37ecfc73734a141979a34f2e155baa40cea5b22b016c192514c9d5847335d949ee642620c4508cc2ec8a2ceeb4dcec381715e633893623c42454975999d1a
7
- data.tar.gz: 2a5f38c78c371e0710354a2b915a074512a87225f33fca95c9088ded76cad0dfc22e454fc9f64bddd40d8b0c0a6ab65bb108f467cd5b2b2c39d651285665f845
6
+ metadata.gz: 74e32eace9001bd50be0617f10234f1440f86e506facc7247cf6041cf81fa85e966a5721e2a7ff364eb89da19bd46cc92b2631d99d4cbb21aee8240a475a76b5
7
+ data.tar.gz: 8e2bc294a1fa08a04f0bdfa300394751e63a7c1a21cac6aa1080a37e9491fe33091150047f664607ae42596d0bb2ac60f05c06ad97aee83165de754b9cceeed2
data/CHANGELOG.md CHANGED
@@ -1,12 +1,34 @@
1
1
  Dalli Changelog
2
2
  =====================
3
3
 
4
- 4.3.2
4
+ 5.0.0
5
5
  ==========
6
6
 
7
+ **Breaking Changes:**
8
+
9
+ - **Removed binary protocol** - The meta protocol is now the only supported protocol
10
+ - The `:protocol` option is no longer used
11
+ - Requires memcached 1.6+ (for meta protocol support)
12
+ - Users on older memcached versions must upgrade or stay on Dalli 4.x
13
+
14
+ - **Removed SASL authentication** - The meta protocol does not support authentication
15
+ - Use network-level security (firewall rules, VPN) or memcached's TLS support instead
16
+ - Users requiring SASL authentication must stay on Dalli 4.x with binary protocol
17
+
18
+ - **Ruby 3.3+ required** - Dropped support for Ruby 3.1 and 3.2
19
+ - Ruby 3.2 reached end-of-life in March 2026
20
+ - JRuby remains supported
21
+
22
+ Performance:
23
+
24
+ - **~7% read performance improvement** (CRuby only)
25
+ - Use native `IO#read` instead of custom `readfull` implementation
26
+ - Enabled by Ruby 3.3's `IO#timeout=` support
27
+ - JRuby continues to use `readfull` for compatibility
28
+
7
29
  OpenTelemetry:
8
30
 
9
- - Migrate to stable OTel semantic conventions
31
+ - Migrate to stable OTel semantic conventions (#1070)
10
32
  - `db.system` renamed to `db.system.name`
11
33
  - `db.operation` renamed to `db.operation.name`
12
34
  - `server.address` now contains hostname only; `server.port` is a separate integer attribute
@@ -16,6 +38,12 @@ OpenTelemetry:
16
38
  - Add `peer.service` span attribute
17
39
  - `:otel_peer_service` option for logical service naming
18
40
 
41
+ Internal:
42
+
43
+ - Simplified protocol directory structure: moved `lib/dalli/protocol/meta/*` to `lib/dalli/protocol/`
44
+ - Removed deprecated binary protocol files and SASL authentication code
45
+ - Removed `require 'set'` (autoloaded in Ruby 3.3+)
46
+
19
47
  4.3.1
20
48
  ==========
21
49
 
data/README.md CHANGED
@@ -10,26 +10,14 @@ Dalli supports:
10
10
  * Fine-grained control of data serialization and compression
11
11
  * Thread-safe operation (either through use of a connection pool, or by using the Dalli client in threadsafe mode)
12
12
  * SSL/TLS connections to memcached
13
- * SASL authentication
14
13
  * OpenTelemetry distributed tracing (automatic when SDK is present)
15
14
 
16
15
  The name is a variant of Salvador Dali for his famous painting [The Persistence of Memory](http://en.wikipedia.org/wiki/The_Persistence_of_Memory).
17
16
 
18
17
  ## Requirements
19
18
 
20
- * Ruby 3.1 or later
21
- * memcached 1.4 or later (1.6+ recommended for meta protocol support)
22
-
23
- ## Protocol Options
24
-
25
- Dalli supports two protocols for communicating with memcached:
26
-
27
- * `:binary` (default) - Works with all memcached versions, supports SASL authentication
28
- * `:meta` - Requires memcached 1.6+, better performance for some operations, no authentication support
29
-
30
- ```ruby
31
- Dalli::Client.new('localhost:11211', protocol: :meta)
32
- ```
19
+ * Ruby 3.3 or later (JRuby also supported)
20
+ * memcached 1.6 or later
33
21
 
34
22
  ## Configuration Options
35
23
 
@@ -64,7 +52,7 @@ By default, Dalli uses Ruby's Marshal for serialization. Deserializing untrusted
64
52
  Dalli::Client.new('localhost:11211', serializer: JSON)
65
53
  ```
66
54
 
67
- See the [4.0-Upgrade.md](4.0-Upgrade.md) guide for more information.
55
+ See the [5.0-Upgrade.md](5.0-Upgrade.md) guide for upgrade information.
68
56
 
69
57
  ## OpenTelemetry Tracing
70
58
 
data/lib/dalli/client.rb CHANGED
@@ -1,7 +1,6 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  require 'digest/md5'
4
- require 'set'
5
4
 
6
5
  # encoding: ascii
7
6
  module Dalli
@@ -48,8 +47,6 @@ module Dalli
48
47
  # serialization pipeline.
49
48
  # - :digest_class - defaults to Digest::MD5, allows you to pass in an object that responds to the hexdigest method,
50
49
  # useful for injecting a FIPS compliant hash object.
51
- # - :protocol - one of either :binary or :meta, defaulting to :binary. This sets the protocol that Dalli uses
52
- # to communicate with memcached.
53
50
  # - :otel_db_statement - controls the +db.query.text+ span attribute when OpenTelemetry is loaded.
54
51
  # +:include+ logs the full operation and key(s), +:obfuscate+ replaces keys with "?",
55
52
  # +nil+ (default) omits the attribute entirely.
@@ -58,9 +55,9 @@ module Dalli
58
55
  def initialize(servers = nil, options = {})
59
56
  @normalized_servers = ::Dalli::ServersArgNormalizer.normalize_servers(servers)
60
57
  @options = normalize_options(options)
58
+ warn_removed_options(@options)
61
59
  @key_manager = ::Dalli::KeyManager.new(@options)
62
60
  @ring = nil
63
- emit_deprecation_warnings
64
61
  end
65
62
 
66
63
  #
@@ -102,10 +99,7 @@ module Dalli
102
99
  end
103
100
 
104
101
  ##
105
- # Get value with extended metadata using the meta protocol.
106
- #
107
- # IMPORTANT: This method requires memcached 1.6+ and the meta protocol (protocol: :meta).
108
- # It will raise an error if used with the binary protocol.
102
+ # Get value with extended metadata.
109
103
  #
110
104
  # @param key [String] the cache key
111
105
  # @param options [Hash] options controlling what metadata to return
@@ -133,8 +127,6 @@ module Dalli
133
127
  # # => { value: "data", cas: 123, hit_before: true, last_access: 42 }
134
128
  #
135
129
  def get_with_metadata(key, options = {})
136
- raise_unless_meta_protocol!
137
-
138
130
  key = key.to_s
139
131
  key = @key_manager.validate_key(key)
140
132
 
@@ -208,9 +200,6 @@ module Dalli
208
200
  # cache entry (the "thundering herd" problem). Only one client wins the right to
209
201
  # regenerate; other clients receive the stale value (if available) or wait.
210
202
  #
211
- # IMPORTANT: This method requires memcached 1.6+ and the meta protocol (protocol: :meta).
212
- # It will raise an error if used with the binary protocol.
213
- #
214
203
  # @param key [String] the cache key
215
204
  # @param ttl [Integer] time-to-live for the cached value in seconds
216
205
  # @param lock_ttl [Integer] how long the lock/stub lives (default: 30 seconds)
@@ -236,8 +225,6 @@ module Dalli
236
225
  def fetch_with_lock(key, ttl: nil, lock_ttl: 30, recache_threshold: nil, req_options: nil, &block)
237
226
  raise ArgumentError, 'Block is required for fetch_with_lock' unless block_given?
238
227
 
239
- raise_unless_meta_protocol!
240
-
241
228
  key = key.to_s
242
229
  key = @key_manager.validate_key(key)
243
230
 
@@ -514,10 +501,6 @@ module Dalli
514
501
 
515
502
  private
516
503
 
517
- # Records hit/miss metrics on a span for cache observability.
518
- # @param span [OpenTelemetry::Trace::Span, nil] the span to record on
519
- # @param key_count [Integer] total keys requested
520
- # @param hit_count [Integer] keys found in cache
521
504
  def record_hit_miss_metrics(span, key_count, hit_count)
522
505
  return unless span
523
506
 
@@ -607,16 +590,7 @@ module Dalli
607
590
  end
608
591
 
609
592
  def ring
610
- @ring ||= Dalli::Ring.new(@normalized_servers, protocol_implementation, @options)
611
- end
612
-
613
- def protocol_implementation
614
- @protocol_implementation ||= case @options[:protocol]&.to_s
615
- when 'meta'
616
- Dalli::Protocol::Meta
617
- else
618
- Dalli::Protocol::Binary
619
- end
593
+ @ring ||= Dalli::Ring.new(@normalized_servers, @options)
620
594
  end
621
595
 
622
596
  ##
@@ -627,7 +601,7 @@ module Dalli
627
601
  #
628
602
  # This method also forces retries on network errors - when
629
603
  # a particular memcached instance becomes unreachable, or the
630
- # operational times out.
604
+ # operation times out.
631
605
  ##
632
606
  def perform(*all_args)
633
607
  return yield if block_given?
@@ -654,6 +628,21 @@ module Dalli
654
628
  raise ArgumentError, "cannot convert :expires_in => #{opts[:expires_in].inspect} to an integer"
655
629
  end
656
630
 
631
+ REMOVED_OPTIONS = {
632
+ protocol: 'Dalli 5.0 only supports the meta protocol. The :protocol option has been removed.',
633
+ username: 'Dalli 5.0 removed SASL authentication support. The :username option is ignored.',
634
+ password: 'Dalli 5.0 removed SASL authentication support. The :password option is ignored.'
635
+ }.freeze
636
+ private_constant :REMOVED_OPTIONS
637
+
638
+ def warn_removed_options(opts)
639
+ REMOVED_OPTIONS.each do |key, message|
640
+ next unless opts.key?(key)
641
+
642
+ Dalli.logger.warn(message)
643
+ end
644
+ end
645
+
657
646
  def pipelined_getter
658
647
  PipelinedGetter.new(ring, @key_manager)
659
648
  end
@@ -665,15 +654,5 @@ module Dalli
665
654
  def pipelined_deleter
666
655
  PipelinedDeleter.new(ring, @key_manager)
667
656
  end
668
-
669
- def raise_unless_meta_protocol!
670
- return if protocol_implementation == Dalli::Protocol::Meta
671
-
672
- raise Dalli::DalliError,
673
- 'This operation requires the meta protocol (memcached 1.6+). ' \
674
- 'Use protocol: :meta when creating the client.'
675
- end
676
-
677
- include ProtocolDeprecations
678
657
  end
679
658
  end
@@ -8,7 +8,7 @@ module Dalli
8
8
  # When OpenTelemetry is loaded, Dalli automatically creates spans for cache operations.
9
9
  # When OpenTelemetry is not available, all tracing methods are no-ops with zero overhead.
10
10
  #
11
- # Dalli 4.3.2 uses the stable OTel semantic conventions for database spans.
11
+ # Dalli 5.0 uses the stable OTel semantic conventions for database spans.
12
12
  #
13
13
  # == Span Attributes
14
14
  #
data/lib/dalli/options.rb CHANGED
@@ -6,7 +6,7 @@ module Dalli
6
6
  # Make Dalli threadsafe by using a lock around all
7
7
  # public server methods.
8
8
  #
9
- # Dalli::Protocol::Binary.extend(Dalli::Threadsafe)
9
+ # Dalli::Protocol::Meta.extend(Dalli::Threadsafe)
10
10
  #
11
11
  module Threadsafe
12
12
  def self.extended(obj)
@@ -1,7 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require 'set'
4
-
5
3
  module Dalli
6
4
  ##
7
5
  # Contains logic for the pipelined gets implemented by the client.
@@ -22,6 +22,7 @@ module Dalli
22
22
 
23
23
  def initialize(attribs, client_options = {})
24
24
  hostname, port, socket_type, @weight, user_creds = ServerConfigParser.parse(attribs)
25
+ warn_uri_credentials(user_creds)
25
26
  @options = client_options.merge(user_creds)
26
27
  @raw_mode = client_options[:raw]
27
28
  @value_marshaller = @raw_mode ? StringMarshaller.new(@options) : ValueMarshaller.new(@options)
@@ -141,18 +142,6 @@ module Dalli
141
142
  !response_buffer.in_progress?
142
143
  end
143
144
 
144
- def username
145
- @options[:username] || ENV.fetch('MEMCACHE_USERNAME', nil)
146
- end
147
-
148
- def password
149
- @options[:password] || ENV.fetch('MEMCACHE_PASSWORD', nil)
150
- end
151
-
152
- def require_auth?
153
- !username.nil?
154
- end
155
-
156
145
  def quiet?
157
146
  Thread.current[::Dalli::QUIET]
158
147
  end
@@ -162,6 +151,16 @@ module Dalli
162
151
 
163
152
  private
164
153
 
154
+ URI_CREDENTIAL_WARNING = 'Dalli 5.0 removed SASL authentication. ' \
155
+ 'Credentials in memcached:// URIs are ignored.'
156
+ private_constant :URI_CREDENTIAL_WARNING
157
+
158
+ def warn_uri_credentials(user_creds)
159
+ return if user_creds[:username].nil? && user_creds[:password].nil?
160
+
161
+ Dalli.logger.warn(URI_CREDENTIAL_WARNING)
162
+ end
163
+
165
164
  ALLOWED_QUIET_OPS = %i[add replace set delete incr decr append prepend flush noop].freeze
166
165
  private_constant :ALLOWED_QUIET_OPS
167
166
 
@@ -216,8 +215,7 @@ module Dalli
216
215
 
217
216
  def connect
218
217
  @connection_manager.establish_connection
219
- authenticate_connection if require_auth?
220
- @version = version # Connect socket if not authed
218
+ @version = version
221
219
  up!
222
220
  end
223
221
 
@@ -156,7 +156,13 @@ module Dalli
156
156
  end
157
157
 
158
158
  def read(count)
159
- @sock.readfull(count)
159
+ # JRuby doesn't support IO#timeout=, so use custom readfull implementation
160
+ # CRuby 3.3+ has IO#timeout= which makes IO#read work with timeouts
161
+ if RUBY_ENGINE == 'jruby'
162
+ @sock.readfull(count)
163
+ else
164
+ @sock.read(count)
165
+ end
160
166
  rescue SystemCallError, *TIMEOUT_ERRORS, *SSL_ERRORS, EOFError => e
161
167
  error_on_request!(e)
162
168
  end
@@ -257,13 +257,9 @@ module Dalli
257
257
  @connection_manager.flush
258
258
  end
259
259
 
260
- def authenticate_connection
261
- raise Dalli::DalliError, 'Authentication not supported for the meta protocol.'
262
- end
263
-
264
- require_relative 'meta/key_regularizer'
265
- require_relative 'meta/request_formatter'
266
- require_relative 'meta/response_processor'
260
+ require_relative 'key_regularizer'
261
+ require_relative 'request_formatter'
262
+ require_relative 'response_processor'
267
263
  end
268
264
  end
269
265
  end
@@ -6,7 +6,7 @@ module Dalli
6
6
  module Protocol
7
7
  ##
8
8
  # Dalli::Protocol::ServerConfigParser parses a server string passed to
9
- # a Dalli::Protocol::Binary instance into the hostname, port, weight, and
9
+ # a Dalli::Protocol::Meta instance into the hostname, port, weight, and
10
10
  # socket_type.
11
11
  ##
12
12
  class ServerConfigParser
data/lib/dalli/ring.rb CHANGED
@@ -23,9 +23,9 @@ module Dalli
23
23
 
24
24
  attr_accessor :servers, :continuum
25
25
 
26
- def initialize(servers_arg, protocol_implementation, options)
26
+ def initialize(servers_arg, options)
27
27
  @servers = servers_arg.map do |s|
28
- protocol_implementation.new(s, options)
28
+ Dalli::Protocol::Meta.new(s, options)
29
29
  end
30
30
  @continuum = nil
31
31
  @continuum = build_continuum(servers) if servers.size > 1
@@ -16,7 +16,7 @@ module Dalli
16
16
  # weight are optional (e.g. 'localhost', 'abc.com:12345', 'example.org:22222:3')
17
17
  # * A colon separated string of (UNIX socket, weight) where the weight is optional
18
18
  # (e.g. '/var/run/memcached/socket', '/tmp/xyz:3') (not supported on Windows)
19
- # * A URI with a 'memcached' protocol, which will typically include a username/password
19
+ # * A URI with a 'memcached' protocol (e.g. 'memcached://localhost:11211')
20
20
  #
21
21
  # The methods in this module do not validate the format of individual server strings, but
22
22
  # rather normalize the argument into a compact array, wherein each array entry corresponds
data/lib/dalli/version.rb CHANGED
@@ -1,7 +1,7 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Dalli
4
- VERSION = '4.3.2'
4
+ VERSION = '5.0.0'
5
5
 
6
- MIN_SUPPORTED_MEMCACHED_VERSION = '1.4'
6
+ MIN_SUPPORTED_MEMCACHED_VERSION = '1.6'
7
7
  end
data/lib/dalli.rb CHANGED
@@ -62,7 +62,6 @@ require_relative 'dalli/version'
62
62
  require_relative 'dalli/instrumentation'
63
63
 
64
64
  require_relative 'dalli/compressor'
65
- require_relative 'dalli/protocol_deprecations'
66
65
  require_relative 'dalli/client'
67
66
  require_relative 'dalli/key_manager'
68
67
  require_relative 'dalli/pipelined_getter'
@@ -71,7 +70,6 @@ require_relative 'dalli/pipelined_deleter'
71
70
  require_relative 'dalli/ring'
72
71
  require_relative 'dalli/protocol'
73
72
  require_relative 'dalli/protocol/base'
74
- require_relative 'dalli/protocol/binary'
75
73
  require_relative 'dalli/protocol/connection_manager'
76
74
  require_relative 'dalli/protocol/meta'
77
75
  require_relative 'dalli/protocol/response_buffer'
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: dalli
3
3
  version: !ruby/object:Gem::Version
4
- version: 4.3.2
4
+ version: 5.0.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Peter M. Goldstein
@@ -49,24 +49,18 @@ files:
49
49
  - lib/dalli/pipelined_setter.rb
50
50
  - lib/dalli/protocol.rb
51
51
  - lib/dalli/protocol/base.rb
52
- - lib/dalli/protocol/binary.rb
53
- - lib/dalli/protocol/binary/request_formatter.rb
54
- - lib/dalli/protocol/binary/response_header.rb
55
- - lib/dalli/protocol/binary/response_processor.rb
56
- - lib/dalli/protocol/binary/sasl_authentication.rb
57
52
  - lib/dalli/protocol/connection_manager.rb
53
+ - lib/dalli/protocol/key_regularizer.rb
58
54
  - lib/dalli/protocol/meta.rb
59
- - lib/dalli/protocol/meta/key_regularizer.rb
60
- - lib/dalli/protocol/meta/request_formatter.rb
61
- - lib/dalli/protocol/meta/response_processor.rb
55
+ - lib/dalli/protocol/request_formatter.rb
62
56
  - lib/dalli/protocol/response_buffer.rb
57
+ - lib/dalli/protocol/response_processor.rb
63
58
  - lib/dalli/protocol/server_config_parser.rb
64
59
  - lib/dalli/protocol/string_marshaller.rb
65
60
  - lib/dalli/protocol/ttl_sanitizer.rb
66
61
  - lib/dalli/protocol/value_compressor.rb
67
62
  - lib/dalli/protocol/value_marshaller.rb
68
63
  - lib/dalli/protocol/value_serializer.rb
69
- - lib/dalli/protocol_deprecations.rb
70
64
  - lib/dalli/ring.rb
71
65
  - lib/dalli/servers_arg_normalizer.rb
72
66
  - lib/dalli/socket.rb
@@ -86,7 +80,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
86
80
  requirements:
87
81
  - - ">="
88
82
  - !ruby/object:Gem::Version
89
- version: '3.1'
83
+ version: '3.3'
90
84
  required_rubygems_version: !ruby/object:Gem::Requirement
91
85
  requirements:
92
86
  - - ">="
@@ -1,117 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- module Dalli
4
- module Protocol
5
- class Binary
6
- ##
7
- # Class that encapsulates logic for formatting binary protocol requests
8
- # to memcached.
9
- ##
10
- class RequestFormatter
11
- REQUEST = 0x80
12
-
13
- OPCODES = {
14
- get: 0x00,
15
- set: 0x01,
16
- add: 0x02,
17
- replace: 0x03,
18
- delete: 0x04,
19
- incr: 0x05,
20
- decr: 0x06,
21
- flush: 0x08,
22
- noop: 0x0A,
23
- version: 0x0B,
24
- getkq: 0x0D,
25
- append: 0x0E,
26
- prepend: 0x0F,
27
- stat: 0x10,
28
- setq: 0x11,
29
- addq: 0x12,
30
- replaceq: 0x13,
31
- deleteq: 0x14,
32
- incrq: 0x15,
33
- decrq: 0x16,
34
- flushq: 0x18,
35
- appendq: 0x19,
36
- prependq: 0x1A,
37
- touch: 0x1C,
38
- gat: 0x1D,
39
- auth_negotiation: 0x20,
40
- auth_request: 0x21,
41
- auth_continue: 0x22
42
- }.freeze
43
-
44
- REQ_HEADER_FORMAT = 'CCnCCnNNQ'
45
-
46
- KEY_ONLY = 'a*'
47
- TTL_AND_KEY = 'Na*'
48
- KEY_AND_VALUE = 'a*a*'
49
- INCR_DECR = 'NNNNNa*'
50
- TTL_ONLY = 'N'
51
- NO_BODY = ''
52
-
53
- BODY_FORMATS = {
54
- get: KEY_ONLY,
55
- getkq: KEY_ONLY,
56
- delete: KEY_ONLY,
57
- deleteq: KEY_ONLY,
58
- stat: KEY_ONLY,
59
-
60
- append: KEY_AND_VALUE,
61
- prepend: KEY_AND_VALUE,
62
- appendq: KEY_AND_VALUE,
63
- prependq: KEY_AND_VALUE,
64
- auth_request: KEY_AND_VALUE,
65
- auth_continue: KEY_AND_VALUE,
66
-
67
- set: 'NNa*a*',
68
- setq: 'NNa*a*',
69
- add: 'NNa*a*',
70
- addq: 'NNa*a*',
71
- replace: 'NNa*a*',
72
- replaceq: 'NNa*a*',
73
-
74
- incr: INCR_DECR,
75
- decr: INCR_DECR,
76
- incrq: INCR_DECR,
77
- decrq: INCR_DECR,
78
-
79
- flush: TTL_ONLY,
80
- flushq: TTL_ONLY,
81
-
82
- noop: NO_BODY,
83
- auth_negotiation: NO_BODY,
84
- version: NO_BODY,
85
-
86
- touch: TTL_AND_KEY,
87
- gat: TTL_AND_KEY
88
- }.freeze
89
- FORMAT = BODY_FORMATS.transform_values { |v| REQ_HEADER_FORMAT + v }
90
-
91
- # rubocop:disable Metrics/ParameterLists
92
- def self.standard_request(opkey:, key: nil, value: nil, opaque: 0, cas: 0, bitflags: nil, ttl: nil)
93
- extra_len = (bitflags.nil? ? 0 : 4) + (ttl.nil? ? 0 : 4)
94
- key_len = key.nil? ? 0 : key.bytesize
95
- value_len = value.nil? ? 0 : value.bytesize
96
- header = [REQUEST, OPCODES[opkey], key_len, extra_len, 0, 0, extra_len + key_len + value_len, opaque, cas]
97
- body = [bitflags, ttl, key, value].compact
98
- (header + body).pack(FORMAT[opkey])
99
- end
100
- # rubocop:enable Metrics/ParameterLists
101
-
102
- def self.decr_incr_request(opkey:, key: nil, count: nil, initial: nil, expiry: nil)
103
- extra_len = 20
104
- (h, l) = as_8byte_uint(count)
105
- (dh, dl) = as_8byte_uint(initial)
106
- header = [REQUEST, OPCODES[opkey], key.bytesize, extra_len, 0, 0, key.bytesize + extra_len, 0, 0]
107
- body = [h, l, dh, dl, expiry, key]
108
- (header + body).pack(FORMAT[opkey])
109
- end
110
-
111
- def self.as_8byte_uint(val)
112
- [val >> 32, val & 0xFFFFFFFF]
113
- end
114
- end
115
- end
116
- end
117
- end
@@ -1,36 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- module Dalli
4
- module Protocol
5
- class Binary
6
- ##
7
- # Class that encapsulates data parsed from a memcached response header.
8
- ##
9
- class ResponseHeader
10
- SIZE = 24
11
- FMT = '@2nCCnNNQ'
12
-
13
- attr_reader :key_len, :extra_len, :data_type, :status, :body_len, :opaque, :cas
14
-
15
- def initialize(buf)
16
- raise ArgumentError, "Response buffer must be at least #{SIZE} bytes" unless buf.bytesize >= SIZE
17
-
18
- @key_len, @extra_len, @data_type, @status, @body_len, @opaque, @cas = buf.unpack(FMT)
19
- end
20
-
21
- def ok?
22
- status.zero?
23
- end
24
-
25
- def not_found?
26
- status == 1
27
- end
28
-
29
- NOT_STORED_STATUSES = [2, 5].freeze
30
- def not_stored?
31
- NOT_STORED_STATUSES.include?(status)
32
- end
33
- end
34
- end
35
- end
36
- end
@@ -1,239 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- module Dalli
4
- module Protocol
5
- class Binary
6
- ##
7
- # Class that encapsulates logic for processing binary protocol responses
8
- # from memcached. Includes logic for pulling data from an IO source
9
- # and parsing into local values. Handles errors on unexpected values.
10
- ##
11
- class ResponseProcessor
12
- # Response codes taken from:
13
- # https://github.com/memcached/memcached/wiki/BinaryProtocolRevamped#response-status
14
- RESPONSE_CODES = {
15
- 0 => 'No error',
16
- 1 => 'Key not found',
17
- 2 => 'Key exists',
18
- 3 => 'Value too large',
19
- 4 => 'Invalid arguments',
20
- 5 => 'Item not stored',
21
- 6 => 'Incr/decr on a non-numeric value',
22
- 7 => 'The vbucket belongs to another server',
23
- 8 => 'Authentication error',
24
- 9 => 'Authentication continue',
25
- 0x20 => 'Authentication required',
26
- 0x81 => 'Unknown command',
27
- 0x82 => 'Out of memory',
28
- 0x83 => 'Not supported',
29
- 0x84 => 'Internal error',
30
- 0x85 => 'Busy',
31
- 0x86 => 'Temporary failure'
32
- }.freeze
33
-
34
- def initialize(io_source, value_marshaller)
35
- @io_source = io_source
36
- @value_marshaller = value_marshaller
37
- end
38
-
39
- def read(num_bytes)
40
- @io_source.read(num_bytes)
41
- end
42
-
43
- def read_response
44
- resp_header = ResponseHeader.new(read_header)
45
- body = read(resp_header.body_len) if resp_header.body_len.positive?
46
- [resp_header, body]
47
- end
48
-
49
- def unpack_response_body(resp_header, body, parse_as_stored_value)
50
- extra_len = resp_header.extra_len
51
- key_len = resp_header.key_len
52
- bitflags = extra_len.positive? ? body.unpack1('N') : 0x0
53
- key = body.byteslice(extra_len, key_len).force_encoding(Encoding::UTF_8) if key_len.positive?
54
- value = body.byteslice((extra_len + key_len)..-1)
55
- value = @value_marshaller.retrieve(value, bitflags) if parse_as_stored_value
56
- [key, value]
57
- end
58
-
59
- def read_header
60
- read(ResponseHeader::SIZE) || raise(Dalli::NetworkError, 'No response')
61
- end
62
-
63
- def raise_on_not_ok!(resp_header)
64
- return if resp_header.ok?
65
-
66
- raise Dalli::DalliError, "Response error #{resp_header.status}: #{RESPONSE_CODES[resp_header.status]}"
67
- end
68
-
69
- def get(cache_nils: false)
70
- resp_header, body = read_response
71
-
72
- return false if resp_header.not_stored? # Not stored, normal status for add operation
73
- return cache_nils ? ::Dalli::NOT_FOUND : nil if resp_header.not_found?
74
-
75
- raise_on_not_ok!(resp_header)
76
- return true unless body
77
-
78
- unpack_response_body(resp_header, body, true).last
79
- end
80
-
81
- ##
82
- # Response for a storage operation. Returns the cas on success. False
83
- # if the value wasn't stored. And raises an error on all other error
84
- # codes from memcached.
85
- ##
86
- def storage_response
87
- resp_header, = read_response
88
- return nil if resp_header.not_found?
89
- return false if resp_header.not_stored? # Not stored, normal status for add operation
90
-
91
- raise_on_not_ok!(resp_header)
92
- resp_header.cas
93
- end
94
-
95
- def delete
96
- resp_header, = read_response
97
- return false if resp_header.not_found? || resp_header.not_stored?
98
-
99
- raise_on_not_ok!(resp_header)
100
- true
101
- end
102
-
103
- def data_cas_response
104
- resp_header, body = read_response
105
- return [nil, resp_header.cas] if resp_header.not_found?
106
- return [nil, false] if resp_header.not_stored?
107
-
108
- raise_on_not_ok!(resp_header)
109
- return [nil, resp_header.cas] unless body
110
-
111
- [unpack_response_body(resp_header, body, true).last, resp_header.cas]
112
- end
113
-
114
- # Returns the new value for the key, if found and updated
115
- def decr_incr
116
- body = generic_response
117
- body ? body.unpack1('Q>') : body
118
- end
119
-
120
- def stats
121
- hash = {}
122
- loop do
123
- resp_header, body = read_response
124
- # This is the response to the terminating noop / end of stat
125
- return hash if resp_header.ok? && resp_header.key_len.zero?
126
-
127
- # Ignore any responses with non-zero status codes,
128
- # such as errors from set operations. That allows
129
- # this code to be used at the end of a multi
130
- # block to clear any error responses from inside the multi.
131
- next unless resp_header.ok?
132
-
133
- key, value = unpack_response_body(resp_header, body, true)
134
- hash[key] = value
135
- end
136
- end
137
-
138
- def flush
139
- no_body_response
140
- end
141
-
142
- def reset
143
- generic_response
144
- end
145
-
146
- def version
147
- generic_response
148
- end
149
-
150
- def consume_all_responses_until_noop
151
- loop do
152
- resp_header, = read_response
153
- # This is the response to the terminating noop / end of stat
154
- return true if resp_header.ok? && resp_header.key_len.zero?
155
- end
156
- end
157
-
158
- def generic_response
159
- resp_header, body = read_response
160
-
161
- return false if resp_header.not_stored? # Not stored, normal status for add operation
162
- return nil if resp_header.not_found?
163
-
164
- raise_on_not_ok!(resp_header)
165
- return true unless body
166
-
167
- unpack_response_body(resp_header, body, false).last
168
- end
169
-
170
- def no_body_response
171
- resp_header, = read_response
172
- return false if resp_header.not_stored? # Not stored, possible status for append/prepend/delete
173
-
174
- raise_on_not_ok!(resp_header)
175
- true
176
- end
177
-
178
- def validate_auth_format(extra_len, count)
179
- return if extra_len.zero?
180
-
181
- raise Dalli::NetworkError, "Unexpected message format: #{extra_len} #{count}"
182
- end
183
-
184
- def auth_response(buf = read_header)
185
- resp_header = ResponseHeader.new(buf)
186
- body_len = resp_header.body_len
187
- validate_auth_format(resp_header.extra_len, body_len)
188
- content = read(body_len) if body_len.positive?
189
- [resp_header.status, content]
190
- end
191
-
192
- def contains_header?(buf)
193
- return false unless buf
194
-
195
- buf.bytesize >= ResponseHeader::SIZE
196
- end
197
-
198
- def response_header_from_buffer(buf)
199
- ResponseHeader.new(buf)
200
- end
201
-
202
- ##
203
- # This method returns an array of values used in a pipelined
204
- # getk process. The first value is the number of bytes by
205
- # which to advance the pointer in the buffer. If the
206
- # complete response is found in the buffer, this will
207
- # be the response size. Otherwise it is zero.
208
- #
209
- # The remaining three values in the array are the ResponseHeader,
210
- # key, and value.
211
- ##
212
- def getk_response_from_buffer(buf)
213
- # There's no header in the buffer, so don't advance
214
- return [0, nil, nil, nil, nil] unless contains_header?(buf)
215
-
216
- resp_header = response_header_from_buffer(buf)
217
- body_len = resp_header.body_len
218
-
219
- # We have a complete response that has no body.
220
- # This is either the response to the terminating
221
- # noop or, if the status is not zero, an intermediate
222
- # error response that needs to be discarded.
223
- return [ResponseHeader::SIZE, resp_header.ok?, resp_header.cas, nil, nil] if body_len.zero?
224
-
225
- resp_size = ResponseHeader::SIZE + body_len
226
- # The header is in the buffer, but the body is not. As we don't have
227
- # a complete response, don't advance the buffer
228
- return [0, nil, nil, nil, nil] unless buf.bytesize >= resp_size
229
-
230
- # The full response is in our buffer, so parse it and return
231
- # the values
232
- body = buf.byteslice(ResponseHeader::SIZE, body_len)
233
- key, value = unpack_response_body(resp_header, body, true)
234
- [resp_size, resp_header.ok?, resp_header.cas, key, value]
235
- end
236
- end
237
- end
238
- end
239
- end
@@ -1,60 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- module Dalli
4
- module Protocol
5
- class Binary
6
- ##
7
- # Code to support SASL authentication
8
- ##
9
- module SaslAuthentication
10
- def perform_auth_negotiation
11
- write(RequestFormatter.standard_request(opkey: :auth_negotiation))
12
-
13
- status, content = response_processor.auth_response
14
- return [status, []] if content.nil?
15
-
16
- # Substitute spaces for the \x00 returned by
17
- # memcached as a separator for easier
18
- content&.tr!("\u0000", ' ')
19
- mechanisms = content&.split
20
- [status, mechanisms]
21
- end
22
-
23
- PLAIN_AUTH = 'PLAIN'
24
-
25
- def supported_mechanisms!(mechanisms)
26
- unless mechanisms.include?(PLAIN_AUTH)
27
- raise NotImplementedError,
28
- 'Dalli only supports the PLAIN authentication mechanism'
29
- end
30
- [PLAIN_AUTH]
31
- end
32
-
33
- def authenticate_with_plain
34
- write(RequestFormatter.standard_request(opkey: :auth_request,
35
- key: PLAIN_AUTH,
36
- value: "\x0#{username}\x0#{password}"))
37
- @response_processor.auth_response
38
- end
39
-
40
- def authenticate_connection
41
- Dalli.logger.info { "Dalli/SASL authenticating as #{username}" }
42
-
43
- status, mechanisms = perform_auth_negotiation
44
- return Dalli.logger.debug('Authentication not required/supported by server') if status == 0x81
45
-
46
- supported_mechanisms!(mechanisms)
47
- status, content = authenticate_with_plain
48
-
49
- return Dalli.logger.info("Dalli/SASL: #{content}") if status.zero?
50
-
51
- raise Dalli::DalliError, "Error authenticating: 0x#{status.to_s(16)}" unless status == 0x21
52
-
53
- raise NotImplementedError, 'No two-step authentication mechanisms supported'
54
- # (step, msg) = sasl.receive('challenge', content)
55
- # raise Dalli::NetworkError, "Authentication failed" if sasl.failed? || step != 'response'
56
- end
57
- end
58
- end
59
- end
60
- end
@@ -1,200 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- require 'forwardable'
4
- require 'socket'
5
- require 'timeout'
6
-
7
- module Dalli
8
- module Protocol
9
- ##
10
- # Access point for a single Memcached server, accessed via Memcached's binary
11
- # protocol. Contains logic for managing connection state to the server (retries, etc),
12
- # formatting requests to the server, and unpacking responses.
13
- ##
14
- class Binary < Base
15
- def response_processor
16
- @response_processor ||= ResponseProcessor.new(@connection_manager, @value_marshaller)
17
- end
18
-
19
- private
20
-
21
- # Retrieval Commands
22
- def get(key, options = nil)
23
- req = RequestFormatter.standard_request(opkey: :get, key: key)
24
- write(req)
25
- @connection_manager.flush
26
- response_processor.get(cache_nils: cache_nils?(options))
27
- end
28
-
29
- def quiet_get_request(key)
30
- RequestFormatter.standard_request(opkey: :getkq, key: key)
31
- end
32
-
33
- def gat(key, ttl, options = nil)
34
- ttl = TtlSanitizer.sanitize(ttl)
35
- req = RequestFormatter.standard_request(opkey: :gat, key: key, ttl: ttl)
36
- write(req)
37
- @connection_manager.flush
38
- response_processor.get(cache_nils: cache_nils?(options))
39
- end
40
-
41
- def touch(key, ttl)
42
- ttl = TtlSanitizer.sanitize(ttl)
43
- write(RequestFormatter.standard_request(opkey: :touch, key: key, ttl: ttl))
44
- @connection_manager.flush
45
- response_processor.generic_response
46
- end
47
-
48
- # TODO: This is confusing, as there's a cas command in memcached
49
- # and this isn't it. Maybe rename? Maybe eliminate?
50
- def cas(key)
51
- req = RequestFormatter.standard_request(opkey: :get, key: key)
52
- write(req)
53
- @connection_manager.flush
54
- response_processor.data_cas_response
55
- end
56
-
57
- # Storage Commands
58
- def set(key, value, ttl, cas, options)
59
- opkey = quiet? ? :setq : :set
60
- storage_req(opkey, key, value, ttl, cas, options)
61
- end
62
-
63
- # Pipelined set - writes a quiet set request without reading response.
64
- # Used by PipelinedSetter for bulk operations.
65
- def pipelined_set(key, value, ttl, options)
66
- storage_req(:setq, key, value, ttl, 0, options)
67
- end
68
-
69
- def add(key, value, ttl, options)
70
- opkey = quiet? ? :addq : :add
71
- storage_req(opkey, key, value, ttl, 0, options)
72
- end
73
-
74
- def replace(key, value, ttl, cas, options)
75
- opkey = quiet? ? :replaceq : :replace
76
- storage_req(opkey, key, value, ttl, cas, options)
77
- end
78
-
79
- # rubocop:disable Metrics/ParameterLists
80
- def storage_req(opkey, key, value, ttl, cas, options)
81
- (value, bitflags) = @value_marshaller.store(key, value, options)
82
- ttl = TtlSanitizer.sanitize(ttl)
83
-
84
- req = RequestFormatter.standard_request(opkey: opkey, key: key,
85
- value: value, bitflags: bitflags,
86
- ttl: ttl, cas: cas)
87
- write(req)
88
- @connection_manager.flush unless quiet?
89
- response_processor.storage_response unless quiet?
90
- end
91
- # rubocop:enable Metrics/ParameterLists
92
-
93
- def append(key, value)
94
- opkey = quiet? ? :appendq : :append
95
- write_append_prepend opkey, key, value
96
- end
97
-
98
- def prepend(key, value)
99
- opkey = quiet? ? :prependq : :prepend
100
- write_append_prepend opkey, key, value
101
- end
102
-
103
- def write_append_prepend(opkey, key, value)
104
- write(RequestFormatter.standard_request(opkey: opkey, key: key, value: value))
105
- @connection_manager.flush unless quiet?
106
- response_processor.no_body_response unless quiet?
107
- end
108
-
109
- # Delete Commands
110
- def delete(key, cas)
111
- opkey = quiet? ? :deleteq : :delete
112
- req = RequestFormatter.standard_request(opkey: opkey, key: key, cas: cas)
113
- write(req)
114
- @connection_manager.flush unless quiet?
115
- response_processor.delete unless quiet?
116
- end
117
-
118
- # Pipelined delete - writes a quiet delete request without reading response.
119
- # Used by PipelinedDeleter for bulk operations.
120
- def pipelined_delete(key)
121
- req = RequestFormatter.standard_request(opkey: :deleteq, key: key, cas: 0)
122
- write(req)
123
- end
124
-
125
- # Arithmetic Commands
126
- def decr(key, count, ttl, initial)
127
- opkey = quiet? ? :decrq : :decr
128
- decr_incr opkey, key, count, ttl, initial
129
- end
130
-
131
- def incr(key, count, ttl, initial)
132
- opkey = quiet? ? :incrq : :incr
133
- decr_incr opkey, key, count, ttl, initial
134
- end
135
-
136
- # This allows us to special case a nil initial value, and
137
- # handle it differently than a zero. This special value
138
- # for expiry causes memcached to return a not found
139
- # if the key doesn't already exist, rather than
140
- # setting the initial value
141
- NOT_FOUND_EXPIRY = 0xFFFFFFFF
142
- private_constant :NOT_FOUND_EXPIRY
143
-
144
- def decr_incr(opkey, key, count, ttl, initial)
145
- expiry = initial ? TtlSanitizer.sanitize(ttl) : NOT_FOUND_EXPIRY
146
- initial ||= 0
147
- write(RequestFormatter.decr_incr_request(opkey: opkey, key: key,
148
- count: count, initial: initial, expiry: expiry))
149
- @connection_manager.flush unless quiet?
150
- response_processor.decr_incr unless quiet?
151
- end
152
-
153
- # Other Commands
154
- def flush(ttl = 0)
155
- opkey = quiet? ? :flushq : :flush
156
- write(RequestFormatter.standard_request(opkey: opkey, ttl: ttl))
157
- @connection_manager.flush unless quiet?
158
- response_processor.no_body_response unless quiet?
159
- end
160
-
161
- # Noop is a keepalive operation but also used to demarcate the end of a set of pipelined commands.
162
- # We need to read all the responses at once.
163
- def noop
164
- write_noop
165
- response_processor.consume_all_responses_until_noop
166
- end
167
-
168
- def stats(info = '')
169
- req = RequestFormatter.standard_request(opkey: :stat, key: info)
170
- write(req)
171
- @connection_manager.flush
172
- response_processor.stats
173
- end
174
-
175
- def reset_stats
176
- write(RequestFormatter.standard_request(opkey: :stat, key: 'reset'))
177
- @connection_manager.flush
178
- response_processor.reset
179
- end
180
-
181
- def version
182
- write(RequestFormatter.standard_request(opkey: :version))
183
- @connection_manager.flush
184
- response_processor.version
185
- end
186
-
187
- def write_noop
188
- req = RequestFormatter.standard_request(opkey: :noop)
189
- write(req)
190
- @connection_manager.flush
191
- end
192
-
193
- require_relative 'binary/request_formatter'
194
- require_relative 'binary/response_header'
195
- require_relative 'binary/response_processor'
196
- require_relative 'binary/sasl_authentication'
197
- include SaslAuthentication
198
- end
199
- end
200
- end
@@ -1,45 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- module Dalli
4
- ##
5
- # Handles deprecation warnings for protocol and authentication features
6
- # that will be removed in Dalli 5.0.
7
- ##
8
- module ProtocolDeprecations
9
- BINARY_PROTOCOL_DEPRECATION_MESSAGE = <<~MSG.chomp
10
- [DEPRECATION] The binary protocol is deprecated and will be removed in Dalli 5.0. \
11
- Please use `protocol: :meta` instead. The meta protocol requires memcached 1.6+. \
12
- See https://github.com/petergoldstein/dalli for migration details.
13
- MSG
14
-
15
- SASL_AUTH_DEPRECATION_MESSAGE = <<~MSG.chomp
16
- [DEPRECATION] SASL authentication is deprecated and will be removed in Dalli 5.0. \
17
- SASL is only supported by the binary protocol, which is being removed. \
18
- Consider using network-level security (firewall rules, VPN) or memcached's TLS support instead.
19
- MSG
20
-
21
- private
22
-
23
- def emit_deprecation_warnings
24
- emit_binary_protocol_deprecation_warning
25
- emit_sasl_auth_deprecation_warning
26
- end
27
-
28
- def emit_binary_protocol_deprecation_warning
29
- protocol = @options[:protocol]
30
- # Binary is used when protocol is nil, :binary, or 'binary'
31
- return if protocol.to_s == 'meta'
32
-
33
- warn BINARY_PROTOCOL_DEPRECATION_MESSAGE
34
- Dalli.logger.warn(BINARY_PROTOCOL_DEPRECATION_MESSAGE)
35
- end
36
-
37
- def emit_sasl_auth_deprecation_warning
38
- username = @options[:username] || ENV.fetch('MEMCACHE_USERNAME', nil)
39
- return unless username
40
-
41
- warn SASL_AUTH_DEPRECATION_MESSAGE
42
- Dalli.logger.warn(SASL_AUTH_DEPRECATION_MESSAGE)
43
- end
44
- end
45
- end