dalli 4.3.4 → 5.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +23 -25
- data/README.md +3 -15
- data/lib/dalli/client.rb +19 -56
- data/lib/dalli/instrumentation.rb +1 -1
- data/lib/dalli/options.rb +1 -1
- data/lib/dalli/pipelined_getter.rb +14 -11
- data/lib/dalli/protocol/base.rb +17 -30
- data/lib/dalli/protocol/connection_manager.rb +7 -1
- data/lib/dalli/protocol/meta.rb +3 -7
- data/lib/dalli/protocol/{meta/request_formatter.rb → request_formatter.rb} +8 -21
- data/lib/dalli/protocol/response_buffer.rb +12 -27
- data/lib/dalli/protocol/{meta/response_processor.rb → response_processor.rb} +22 -10
- data/lib/dalli/protocol/server_config_parser.rb +1 -1
- data/lib/dalli/ring.rb +2 -2
- data/lib/dalli/servers_arg_normalizer.rb +1 -1
- data/lib/dalli/version.rb +2 -2
- data/lib/dalli.rb +0 -2
- metadata +7 -13
- data/lib/dalli/protocol/binary/request_formatter.rb +0 -117
- data/lib/dalli/protocol/binary/response_header.rb +0 -36
- data/lib/dalli/protocol/binary/response_processor.rb +0 -229
- data/lib/dalli/protocol/binary/sasl_authentication.rb +0 -60
- data/lib/dalli/protocol/binary.rb +0 -200
- data/lib/dalli/protocol_deprecations.rb +0 -45
- /data/lib/dalli/protocol/{meta/key_regularizer.rb → key_regularizer.rb} +0 -0
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 16d7d5950137f4a51ae41392eaa576732595ac4020ae7be972d6fb4b33b9861d
|
|
4
|
+
data.tar.gz: ebb799522259a9a32a86e1dbfd13e810427f787f8110415509288655a5a5460b
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 74e32eace9001bd50be0617f10234f1440f86e506facc7247cf6041cf81fa85e966a5721e2a7ff364eb89da19bd46cc92b2631d99d4cbb21aee8240a475a76b5
|
|
7
|
+
data.tar.gz: 8e2bc294a1fa08a04f0bdfa300394751e63a7c1a21cac6aa1080a37e9491fe33091150047f664607ae42596d0bb2ac60f05c06ad97aee83165de754b9cceeed2
|
data/CHANGELOG.md
CHANGED
|
@@ -1,42 +1,34 @@
|
|
|
1
1
|
Dalli Changelog
|
|
2
2
|
=====================
|
|
3
3
|
|
|
4
|
-
|
|
4
|
+
5.0.0
|
|
5
5
|
==========
|
|
6
6
|
|
|
7
|
-
|
|
8
|
-
==========
|
|
7
|
+
**Breaking Changes:**
|
|
9
8
|
|
|
10
|
-
|
|
9
|
+
- **Removed binary protocol** - The meta protocol is now the only supported protocol
|
|
10
|
+
- The `:protocol` option is no longer used
|
|
11
|
+
- Requires memcached 1.6+ (for meta protocol support)
|
|
12
|
+
- Users on older memcached versions must upgrade or stay on Dalli 4.x
|
|
11
13
|
|
|
12
|
-
-
|
|
13
|
-
-
|
|
14
|
-
-
|
|
15
|
-
- As defense in depth, the meta `RequestFormatter` now converts every numeric flag it writes (`D`, `J`, `N`, `R`, `T`) to an Integer
|
|
16
|
-
- Thanks to oss-security-shop for the report
|
|
14
|
+
- **Removed SASL authentication** - The meta protocol does not support authentication
|
|
15
|
+
- Use network-level security (firewall rules, VPN) or memcached's TLS support instead
|
|
16
|
+
- Users requiring SASL authentication must stay on Dalli 4.x with binary protocol
|
|
17
17
|
|
|
18
|
-
|
|
19
|
-
|
|
18
|
+
- **Ruby 3.3+ required** - Dropped support for Ruby 3.1 and 3.2
|
|
19
|
+
- Ruby 3.2 reached end-of-life in March 2026
|
|
20
|
+
- JRuby remains supported
|
|
20
21
|
|
|
21
22
|
Performance:
|
|
22
23
|
|
|
23
|
-
-
|
|
24
|
-
-
|
|
25
|
-
-
|
|
26
|
-
-
|
|
27
|
-
- `PipelinedGetter`: replace Hash-based socket-to-server mapping with linear scan (faster for typical 1-5 server counts); use `Process.clock_gettime(CLOCK_MONOTONIC)` instead of `Time.now`
|
|
28
|
-
- Add cross-version benchmark script (`bin/compare_versions`) for reproducible performance comparisons across Dalli versions
|
|
29
|
-
|
|
30
|
-
Bug Fixes:
|
|
31
|
-
|
|
32
|
-
- Skip OTel integration tests when meta protocol is unavailable (#1072)
|
|
33
|
-
|
|
34
|
-
4.3.2
|
|
35
|
-
==========
|
|
24
|
+
- **~7% read performance improvement** (CRuby only)
|
|
25
|
+
- Use native `IO#read` instead of custom `readfull` implementation
|
|
26
|
+
- Enabled by Ruby 3.3's `IO#timeout=` support
|
|
27
|
+
- JRuby continues to use `readfull` for compatibility
|
|
36
28
|
|
|
37
29
|
OpenTelemetry:
|
|
38
30
|
|
|
39
|
-
- Migrate to stable OTel semantic conventions
|
|
31
|
+
- Migrate to stable OTel semantic conventions (#1070)
|
|
40
32
|
- `db.system` renamed to `db.system.name`
|
|
41
33
|
- `db.operation` renamed to `db.operation.name`
|
|
42
34
|
- `server.address` now contains hostname only; `server.port` is a separate integer attribute
|
|
@@ -46,6 +38,12 @@ OpenTelemetry:
|
|
|
46
38
|
- Add `peer.service` span attribute
|
|
47
39
|
- `:otel_peer_service` option for logical service naming
|
|
48
40
|
|
|
41
|
+
Internal:
|
|
42
|
+
|
|
43
|
+
- Simplified protocol directory structure: moved `lib/dalli/protocol/meta/*` to `lib/dalli/protocol/`
|
|
44
|
+
- Removed deprecated binary protocol files and SASL authentication code
|
|
45
|
+
- Removed `require 'set'` (autoloaded in Ruby 3.3+)
|
|
46
|
+
|
|
49
47
|
4.3.1
|
|
50
48
|
==========
|
|
51
49
|
|
data/README.md
CHANGED
|
@@ -10,26 +10,14 @@ Dalli supports:
|
|
|
10
10
|
* Fine-grained control of data serialization and compression
|
|
11
11
|
* Thread-safe operation (either through use of a connection pool, or by using the Dalli client in threadsafe mode)
|
|
12
12
|
* SSL/TLS connections to memcached
|
|
13
|
-
* SASL authentication
|
|
14
13
|
* OpenTelemetry distributed tracing (automatic when SDK is present)
|
|
15
14
|
|
|
16
15
|
The name is a variant of Salvador Dali for his famous painting [The Persistence of Memory](http://en.wikipedia.org/wiki/The_Persistence_of_Memory).
|
|
17
16
|
|
|
18
17
|
## Requirements
|
|
19
18
|
|
|
20
|
-
* Ruby 3.
|
|
21
|
-
* memcached 1.
|
|
22
|
-
|
|
23
|
-
## Protocol Options
|
|
24
|
-
|
|
25
|
-
Dalli supports two protocols for communicating with memcached:
|
|
26
|
-
|
|
27
|
-
* `:binary` (default) - Works with all memcached versions, supports SASL authentication
|
|
28
|
-
* `:meta` - Requires memcached 1.6+, better performance for some operations, no authentication support
|
|
29
|
-
|
|
30
|
-
```ruby
|
|
31
|
-
Dalli::Client.new('localhost:11211', protocol: :meta)
|
|
32
|
-
```
|
|
19
|
+
* Ruby 3.3 or later (JRuby also supported)
|
|
20
|
+
* memcached 1.6 or later
|
|
33
21
|
|
|
34
22
|
## Configuration Options
|
|
35
23
|
|
|
@@ -64,7 +52,7 @@ By default, Dalli uses Ruby's Marshal for serialization. Deserializing untrusted
|
|
|
64
52
|
Dalli::Client.new('localhost:11211', serializer: JSON)
|
|
65
53
|
```
|
|
66
54
|
|
|
67
|
-
See the [
|
|
55
|
+
See the [5.0-Upgrade.md](5.0-Upgrade.md) guide for upgrade information.
|
|
68
56
|
|
|
69
57
|
## OpenTelemetry Tracing
|
|
70
58
|
|
data/lib/dalli/client.rb
CHANGED
|
@@ -1,7 +1,6 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
require 'digest/md5'
|
|
4
|
-
require 'set'
|
|
5
4
|
|
|
6
5
|
# encoding: ascii
|
|
7
6
|
module Dalli
|
|
@@ -48,8 +47,6 @@ module Dalli
|
|
|
48
47
|
# serialization pipeline.
|
|
49
48
|
# - :digest_class - defaults to Digest::MD5, allows you to pass in an object that responds to the hexdigest method,
|
|
50
49
|
# useful for injecting a FIPS compliant hash object.
|
|
51
|
-
# - :protocol - one of either :binary or :meta, defaulting to :binary. This sets the protocol that Dalli uses
|
|
52
|
-
# to communicate with memcached.
|
|
53
50
|
# - :otel_db_statement - controls the +db.query.text+ span attribute when OpenTelemetry is loaded.
|
|
54
51
|
# +:include+ logs the full operation and key(s), +:obfuscate+ replaces keys with "?",
|
|
55
52
|
# +nil+ (default) omits the attribute entirely.
|
|
@@ -58,9 +55,9 @@ module Dalli
|
|
|
58
55
|
def initialize(servers = nil, options = {})
|
|
59
56
|
@normalized_servers = ::Dalli::ServersArgNormalizer.normalize_servers(servers)
|
|
60
57
|
@options = normalize_options(options)
|
|
58
|
+
warn_removed_options(@options)
|
|
61
59
|
@key_manager = ::Dalli::KeyManager.new(@options)
|
|
62
60
|
@ring = nil
|
|
63
|
-
emit_deprecation_warnings
|
|
64
61
|
end
|
|
65
62
|
|
|
66
63
|
#
|
|
@@ -102,10 +99,7 @@ module Dalli
|
|
|
102
99
|
end
|
|
103
100
|
|
|
104
101
|
##
|
|
105
|
-
# Get value with extended metadata
|
|
106
|
-
#
|
|
107
|
-
# IMPORTANT: This method requires memcached 1.6+ and the meta protocol (protocol: :meta).
|
|
108
|
-
# It will raise an error if used with the binary protocol.
|
|
102
|
+
# Get value with extended metadata.
|
|
109
103
|
#
|
|
110
104
|
# @param key [String] the cache key
|
|
111
105
|
# @param options [Hash] options controlling what metadata to return
|
|
@@ -133,8 +127,6 @@ module Dalli
|
|
|
133
127
|
# # => { value: "data", cas: 123, hit_before: true, last_access: 42 }
|
|
134
128
|
#
|
|
135
129
|
def get_with_metadata(key, options = {})
|
|
136
|
-
raise_unless_meta_protocol!
|
|
137
|
-
|
|
138
130
|
key = key.to_s
|
|
139
131
|
key = @key_manager.validate_key(key)
|
|
140
132
|
|
|
@@ -208,9 +200,6 @@ module Dalli
|
|
|
208
200
|
# cache entry (the "thundering herd" problem). Only one client wins the right to
|
|
209
201
|
# regenerate; other clients receive the stale value (if available) or wait.
|
|
210
202
|
#
|
|
211
|
-
# IMPORTANT: This method requires memcached 1.6+ and the meta protocol (protocol: :meta).
|
|
212
|
-
# It will raise an error if used with the binary protocol.
|
|
213
|
-
#
|
|
214
203
|
# @param key [String] the cache key
|
|
215
204
|
# @param ttl [Integer] time-to-live for the cached value in seconds
|
|
216
205
|
# @param lock_ttl [Integer] how long the lock/stub lives (default: 30 seconds)
|
|
@@ -236,10 +225,6 @@ module Dalli
|
|
|
236
225
|
def fetch_with_lock(key, ttl: nil, lock_ttl: 30, recache_threshold: nil, req_options: nil, &block)
|
|
237
226
|
raise ArgumentError, 'Block is required for fetch_with_lock' unless block_given?
|
|
238
227
|
|
|
239
|
-
raise_unless_meta_protocol!
|
|
240
|
-
validate_integer!(:lock_ttl, lock_ttl)
|
|
241
|
-
validate_integer!(:recache_threshold, recache_threshold)
|
|
242
|
-
|
|
243
228
|
key = key.to_s
|
|
244
229
|
key = @key_manager.validate_key(key)
|
|
245
230
|
|
|
@@ -404,8 +389,6 @@ module Dalli
|
|
|
404
389
|
##
|
|
405
390
|
# Incr adds the given amount to the counter on the memcached server.
|
|
406
391
|
# Amt must be a positive integer value.
|
|
407
|
-
# Default, if given, must be an Integer (or a String of decimal digits);
|
|
408
|
-
# anything else raises ArgumentError.
|
|
409
392
|
#
|
|
410
393
|
# If default is nil, the counter must already exist or the operation
|
|
411
394
|
# will fail and will return nil. Otherwise this method will return
|
|
@@ -418,7 +401,6 @@ module Dalli
|
|
|
418
401
|
# If the value already exists, it must have been set with raw: true
|
|
419
402
|
def incr(key, amt = 1, ttl = nil, default = nil)
|
|
420
403
|
check_positive!(amt)
|
|
421
|
-
validate_integer!(:default, default)
|
|
422
404
|
|
|
423
405
|
perform(:incr, key, amt.to_i, ttl_or_default(ttl), default)
|
|
424
406
|
end
|
|
@@ -426,8 +408,6 @@ module Dalli
|
|
|
426
408
|
##
|
|
427
409
|
# Decr subtracts the given amount from the counter on the memcached server.
|
|
428
410
|
# Amt must be a positive integer value.
|
|
429
|
-
# Default, if given, must be an Integer (or a String of decimal digits);
|
|
430
|
-
# anything else raises ArgumentError.
|
|
431
411
|
#
|
|
432
412
|
# memcached counters are unsigned and cannot hold negative values. Calling
|
|
433
413
|
# decr on a counter which is 0 will just return 0.
|
|
@@ -443,7 +423,6 @@ module Dalli
|
|
|
443
423
|
# If the value already exists, it must have been set with raw: true
|
|
444
424
|
def decr(key, amt = 1, ttl = nil, default = nil)
|
|
445
425
|
check_positive!(amt)
|
|
446
|
-
validate_integer!(:default, default)
|
|
447
426
|
|
|
448
427
|
perform(:decr, key, amt.to_i, ttl_or_default(ttl), default)
|
|
449
428
|
end
|
|
@@ -522,10 +501,6 @@ module Dalli
|
|
|
522
501
|
|
|
523
502
|
private
|
|
524
503
|
|
|
525
|
-
# Records hit/miss metrics on a span for cache observability.
|
|
526
|
-
# @param span [OpenTelemetry::Trace::Span, nil] the span to record on
|
|
527
|
-
# @param key_count [Integer] total keys requested
|
|
528
|
-
# @param hit_count [Integer] keys found in cache
|
|
529
504
|
def record_hit_miss_metrics(span, key_count, hit_count)
|
|
530
505
|
return unless span
|
|
531
506
|
|
|
@@ -585,14 +560,6 @@ module Dalli
|
|
|
585
560
|
raise ArgumentError, "Positive values only: #{amt}" if amt.negative?
|
|
586
561
|
end
|
|
587
562
|
|
|
588
|
-
# Numeric arguments that become meta protocol flags (GHSA-6wmv-xq9m-fmp7).
|
|
589
|
-
# Checked before the request starts so a bad value raises a clean
|
|
590
|
-
# ArgumentError, instead of Protocol::Base#request logging it as
|
|
591
|
-
# unexpected and closing the connection.
|
|
592
|
-
def validate_integer!(name, value)
|
|
593
|
-
Dalli::Protocol::Meta::RequestFormatter.integer_flag(name, value) unless value.nil?
|
|
594
|
-
end
|
|
595
|
-
|
|
596
563
|
def cas_core(key, always_set, ttl = nil, req_options = nil)
|
|
597
564
|
(value, cas) = perform(:cas, key)
|
|
598
565
|
return if value.nil? && !always_set
|
|
@@ -623,16 +590,7 @@ module Dalli
|
|
|
623
590
|
end
|
|
624
591
|
|
|
625
592
|
def ring
|
|
626
|
-
@ring ||= Dalli::Ring.new(@normalized_servers,
|
|
627
|
-
end
|
|
628
|
-
|
|
629
|
-
def protocol_implementation
|
|
630
|
-
@protocol_implementation ||= case @options[:protocol]&.to_s
|
|
631
|
-
when 'meta'
|
|
632
|
-
Dalli::Protocol::Meta
|
|
633
|
-
else
|
|
634
|
-
Dalli::Protocol::Binary
|
|
635
|
-
end
|
|
593
|
+
@ring ||= Dalli::Ring.new(@normalized_servers, @options)
|
|
636
594
|
end
|
|
637
595
|
|
|
638
596
|
##
|
|
@@ -643,7 +601,7 @@ module Dalli
|
|
|
643
601
|
#
|
|
644
602
|
# This method also forces retries on network errors - when
|
|
645
603
|
# a particular memcached instance becomes unreachable, or the
|
|
646
|
-
#
|
|
604
|
+
# operation times out.
|
|
647
605
|
##
|
|
648
606
|
def perform(*all_args)
|
|
649
607
|
return yield if block_given?
|
|
@@ -670,6 +628,21 @@ module Dalli
|
|
|
670
628
|
raise ArgumentError, "cannot convert :expires_in => #{opts[:expires_in].inspect} to an integer"
|
|
671
629
|
end
|
|
672
630
|
|
|
631
|
+
REMOVED_OPTIONS = {
|
|
632
|
+
protocol: 'Dalli 5.0 only supports the meta protocol. The :protocol option has been removed.',
|
|
633
|
+
username: 'Dalli 5.0 removed SASL authentication support. The :username option is ignored.',
|
|
634
|
+
password: 'Dalli 5.0 removed SASL authentication support. The :password option is ignored.'
|
|
635
|
+
}.freeze
|
|
636
|
+
private_constant :REMOVED_OPTIONS
|
|
637
|
+
|
|
638
|
+
def warn_removed_options(opts)
|
|
639
|
+
REMOVED_OPTIONS.each do |key, message|
|
|
640
|
+
next unless opts.key?(key)
|
|
641
|
+
|
|
642
|
+
Dalli.logger.warn(message)
|
|
643
|
+
end
|
|
644
|
+
end
|
|
645
|
+
|
|
673
646
|
def pipelined_getter
|
|
674
647
|
PipelinedGetter.new(ring, @key_manager)
|
|
675
648
|
end
|
|
@@ -681,15 +654,5 @@ module Dalli
|
|
|
681
654
|
def pipelined_deleter
|
|
682
655
|
PipelinedDeleter.new(ring, @key_manager)
|
|
683
656
|
end
|
|
684
|
-
|
|
685
|
-
def raise_unless_meta_protocol!
|
|
686
|
-
return if protocol_implementation == Dalli::Protocol::Meta
|
|
687
|
-
|
|
688
|
-
raise Dalli::DalliError,
|
|
689
|
-
'This operation requires the meta protocol (memcached 1.6+). ' \
|
|
690
|
-
'Use protocol: :meta when creating the client.'
|
|
691
|
-
end
|
|
692
|
-
|
|
693
|
-
include ProtocolDeprecations
|
|
694
657
|
end
|
|
695
658
|
end
|
|
@@ -8,7 +8,7 @@ module Dalli
|
|
|
8
8
|
# When OpenTelemetry is loaded, Dalli automatically creates spans for cache operations.
|
|
9
9
|
# When OpenTelemetry is not available, all tracing methods are no-ops with zero overhead.
|
|
10
10
|
#
|
|
11
|
-
# Dalli
|
|
11
|
+
# Dalli 5.0 uses the stable OTel semantic conventions for database spans.
|
|
12
12
|
#
|
|
13
13
|
# == Span Attributes
|
|
14
14
|
#
|
data/lib/dalli/options.rb
CHANGED
|
@@ -1,7 +1,5 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
-
require 'set'
|
|
4
|
-
|
|
5
3
|
module Dalli
|
|
6
4
|
##
|
|
7
5
|
# Contains logic for the pipelined gets implemented by the client.
|
|
@@ -29,7 +27,7 @@ module Dalli
|
|
|
29
27
|
# Stores partial results collected during interleaved send phase
|
|
30
28
|
@partial_results = {}
|
|
31
29
|
servers = setup_requests(keys)
|
|
32
|
-
start_time =
|
|
30
|
+
start_time = Time.now
|
|
33
31
|
|
|
34
32
|
# First yield any partial results collected during interleaved send
|
|
35
33
|
yield_partial_results(&block)
|
|
@@ -149,7 +147,7 @@ module Dalli
|
|
|
149
147
|
end
|
|
150
148
|
|
|
151
149
|
def remaining_time(start, timeout)
|
|
152
|
-
elapsed =
|
|
150
|
+
elapsed = Time.now - start
|
|
153
151
|
return 0 if elapsed > timeout
|
|
154
152
|
|
|
155
153
|
timeout - elapsed
|
|
@@ -168,8 +166,8 @@ module Dalli
|
|
|
168
166
|
# Processes responses from a server. Returns true if there are no
|
|
169
167
|
# additional responses from this server.
|
|
170
168
|
def process_server(server)
|
|
171
|
-
server.pipeline_next_responses do |key,
|
|
172
|
-
yield @key_manager.key_without_namespace(key),
|
|
169
|
+
server.pipeline_next_responses.each_pair do |key, value_list|
|
|
170
|
+
yield @key_manager.key_without_namespace(key), value_list
|
|
173
171
|
end
|
|
174
172
|
|
|
175
173
|
server.pipeline_complete?
|
|
@@ -178,13 +176,18 @@ module Dalli
|
|
|
178
176
|
def servers_with_response(servers, timeout)
|
|
179
177
|
return [] if servers.empty?
|
|
180
178
|
|
|
181
|
-
|
|
182
|
-
|
|
179
|
+
# TODO: - This is a bit challenging. Essentially the PipelinedGetter
|
|
180
|
+
# is a reactor, but without the benefit of a Fiber or separate thread.
|
|
181
|
+
# My suspicion is that we may want to try and push this down into the
|
|
182
|
+
# individual servers, but I'm not sure. For now, we keep the
|
|
183
|
+
# mapping between the alerted object (the socket) and the
|
|
184
|
+
# corrresponding server here.
|
|
185
|
+
server_map = servers.each_with_object({}) { |s, h| h[s.sock] = s }
|
|
186
|
+
|
|
187
|
+
readable, = IO.select(server_map.keys, nil, nil, timeout)
|
|
183
188
|
return [] if readable.nil?
|
|
184
189
|
|
|
185
|
-
|
|
186
|
-
# building and looking up a hash map
|
|
187
|
-
readable.filter_map { |sock| servers.find { |s| s.sock == sock } }
|
|
190
|
+
readable.map { |sock| server_map[sock] }
|
|
188
191
|
end
|
|
189
192
|
|
|
190
193
|
def groups_for_keys(*keys)
|
data/lib/dalli/protocol/base.rb
CHANGED
|
@@ -22,6 +22,7 @@ module Dalli
|
|
|
22
22
|
|
|
23
23
|
def initialize(attribs, client_options = {})
|
|
24
24
|
hostname, port, socket_type, @weight, user_creds = ServerConfigParser.parse(attribs)
|
|
25
|
+
warn_uri_credentials(user_creds)
|
|
25
26
|
@options = client_options.merge(user_creds)
|
|
26
27
|
@raw_mode = client_options[:raw]
|
|
27
28
|
@value_marshaller = @raw_mode ? StringMarshaller.new(@options) : ValueMarshaller.new(@options)
|
|
@@ -91,13 +92,10 @@ module Dalli
|
|
|
91
92
|
# repeatedly whenever this server's socket is readable until
|
|
92
93
|
# #pipeline_complete?.
|
|
93
94
|
#
|
|
94
|
-
#
|
|
95
|
-
|
|
96
|
-
# Without a block, returns a Hash of { key => [value, cas] }.
|
|
97
|
-
# rubocop:disable Metrics/AbcSize, Metrics/CyclomaticComplexity, Metrics/PerceivedComplexity
|
|
98
|
-
def pipeline_next_responses(&block)
|
|
95
|
+
# Returns a Hash of kv pairs received.
|
|
96
|
+
def pipeline_next_responses
|
|
99
97
|
reconnect_on_pipeline_complete!
|
|
100
|
-
values =
|
|
98
|
+
values = {}
|
|
101
99
|
|
|
102
100
|
response_buffer.read
|
|
103
101
|
|
|
@@ -111,24 +109,16 @@ module Dalli
|
|
|
111
109
|
|
|
112
110
|
# If the status is ok and the key is not nil, then this is a
|
|
113
111
|
# getkq response with a value that we want to set in the response hash
|
|
114
|
-
unless key.nil?
|
|
115
|
-
if block
|
|
116
|
-
yield key, value, cas
|
|
117
|
-
else
|
|
118
|
-
values ||= {}
|
|
119
|
-
values[key] = [value, cas]
|
|
120
|
-
end
|
|
121
|
-
end
|
|
112
|
+
values[key] = [value, cas] unless key.nil?
|
|
122
113
|
|
|
123
114
|
# Get the next response from the buffer
|
|
124
115
|
status, cas, key, value = response_buffer.process_single_getk_response
|
|
125
116
|
end
|
|
126
117
|
|
|
127
|
-
values
|
|
118
|
+
values
|
|
128
119
|
rescue SystemCallError, *TIMEOUT_ERRORS, *SSL_ERRORS, EOFError => e
|
|
129
120
|
@connection_manager.error_on_request!(e)
|
|
130
121
|
end
|
|
131
|
-
# rubocop:enable Metrics/AbcSize, Metrics/CyclomaticComplexity, Metrics/PerceivedComplexity
|
|
132
122
|
|
|
133
123
|
# Abort current pipelined get. Generally used to signal an external
|
|
134
124
|
# timeout during pipelined get. The underlying socket is
|
|
@@ -152,18 +142,6 @@ module Dalli
|
|
|
152
142
|
!response_buffer.in_progress?
|
|
153
143
|
end
|
|
154
144
|
|
|
155
|
-
def username
|
|
156
|
-
@options[:username] || ENV.fetch('MEMCACHE_USERNAME', nil)
|
|
157
|
-
end
|
|
158
|
-
|
|
159
|
-
def password
|
|
160
|
-
@options[:password] || ENV.fetch('MEMCACHE_PASSWORD', nil)
|
|
161
|
-
end
|
|
162
|
-
|
|
163
|
-
def require_auth?
|
|
164
|
-
!username.nil?
|
|
165
|
-
end
|
|
166
|
-
|
|
167
145
|
def quiet?
|
|
168
146
|
Thread.current[::Dalli::QUIET]
|
|
169
147
|
end
|
|
@@ -173,6 +151,16 @@ module Dalli
|
|
|
173
151
|
|
|
174
152
|
private
|
|
175
153
|
|
|
154
|
+
URI_CREDENTIAL_WARNING = 'Dalli 5.0 removed SASL authentication. ' \
|
|
155
|
+
'Credentials in memcached:// URIs are ignored.'
|
|
156
|
+
private_constant :URI_CREDENTIAL_WARNING
|
|
157
|
+
|
|
158
|
+
def warn_uri_credentials(user_creds)
|
|
159
|
+
return if user_creds[:username].nil? && user_creds[:password].nil?
|
|
160
|
+
|
|
161
|
+
Dalli.logger.warn(URI_CREDENTIAL_WARNING)
|
|
162
|
+
end
|
|
163
|
+
|
|
176
164
|
ALLOWED_QUIET_OPS = %i[add replace set delete incr decr append prepend flush noop].freeze
|
|
177
165
|
private_constant :ALLOWED_QUIET_OPS
|
|
178
166
|
|
|
@@ -227,8 +215,7 @@ module Dalli
|
|
|
227
215
|
|
|
228
216
|
def connect
|
|
229
217
|
@connection_manager.establish_connection
|
|
230
|
-
|
|
231
|
-
@version = version # Connect socket if not authed
|
|
218
|
+
@version = version
|
|
232
219
|
up!
|
|
233
220
|
end
|
|
234
221
|
|
|
@@ -156,7 +156,13 @@ module Dalli
|
|
|
156
156
|
end
|
|
157
157
|
|
|
158
158
|
def read(count)
|
|
159
|
-
|
|
159
|
+
# JRuby doesn't support IO#timeout=, so use custom readfull implementation
|
|
160
|
+
# CRuby 3.3+ has IO#timeout= which makes IO#read work with timeouts
|
|
161
|
+
if RUBY_ENGINE == 'jruby'
|
|
162
|
+
@sock.readfull(count)
|
|
163
|
+
else
|
|
164
|
+
@sock.read(count)
|
|
165
|
+
end
|
|
160
166
|
rescue SystemCallError, *TIMEOUT_ERRORS, *SSL_ERRORS, EOFError => e
|
|
161
167
|
error_on_request!(e)
|
|
162
168
|
end
|
data/lib/dalli/protocol/meta.rb
CHANGED
|
@@ -257,13 +257,9 @@ module Dalli
|
|
|
257
257
|
@connection_manager.flush
|
|
258
258
|
end
|
|
259
259
|
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
|
|
263
|
-
|
|
264
|
-
require_relative 'meta/key_regularizer'
|
|
265
|
-
require_relative 'meta/request_formatter'
|
|
266
|
-
require_relative 'meta/response_processor'
|
|
260
|
+
require_relative 'key_regularizer'
|
|
261
|
+
require_relative 'request_formatter'
|
|
262
|
+
require_relative 'response_processor'
|
|
267
263
|
end
|
|
268
264
|
end
|
|
269
265
|
end
|
|
@@ -45,11 +45,10 @@ module Dalli
|
|
|
45
45
|
cmd << (skip_flags ? ' v' : ' v f') if value
|
|
46
46
|
cmd << ' c' if return_cas
|
|
47
47
|
cmd << ' b' if base64
|
|
48
|
-
cmd << " T#{
|
|
48
|
+
cmd << " T#{ttl}" if ttl
|
|
49
49
|
cmd << ' k q s' if quiet # Return the key in the response if quiet
|
|
50
|
-
cmd << " N#{
|
|
51
|
-
# Thundering herd: win recache if TTL below threshold
|
|
52
|
-
cmd << " R#{integer_flag(:recache_ttl, recache_ttl)}" if recache_ttl
|
|
50
|
+
cmd << " N#{vivify_ttl}" if vivify_ttl # Thundering herd: vivify on miss
|
|
51
|
+
cmd << " R#{recache_ttl}" if recache_ttl # Thundering herd: win recache if TTL below threshold
|
|
53
52
|
cmd << ' h' if return_hit_status # Return hit status (0 or 1)
|
|
54
53
|
cmd << ' l' if return_last_access # Return seconds since last access
|
|
55
54
|
cmd << ' u' if skip_lru_bump # Don't bump LRU or update access stats
|
|
@@ -62,7 +61,7 @@ module Dalli
|
|
|
62
61
|
cmd << ' b' if base64
|
|
63
62
|
cmd << " F#{bitflags}" if bitflags
|
|
64
63
|
cmd << cas_string(cas)
|
|
65
|
-
cmd << " T#{
|
|
64
|
+
cmd << " T#{ttl}" if ttl
|
|
66
65
|
cmd << " M#{mode_to_token(mode)}"
|
|
67
66
|
cmd << ' q' if quiet
|
|
68
67
|
cmd << TERMINATOR
|
|
@@ -75,7 +74,7 @@ module Dalli
|
|
|
75
74
|
cmd = "md #{key}"
|
|
76
75
|
cmd << ' b' if base64
|
|
77
76
|
cmd << cas_string(cas)
|
|
78
|
-
cmd << " T#{
|
|
77
|
+
cmd << " T#{ttl}" if ttl
|
|
79
78
|
cmd << ' I' if stale # Mark stale instead of deleting
|
|
80
79
|
cmd << ' q' if quiet
|
|
81
80
|
cmd + TERMINATOR
|
|
@@ -84,10 +83,10 @@ module Dalli
|
|
|
84
83
|
def self.meta_arithmetic(key:, delta:, initial:, incr: true, cas: nil, ttl: nil, base64: false, quiet: false)
|
|
85
84
|
cmd = "ma #{key} v"
|
|
86
85
|
cmd << ' b' if base64
|
|
87
|
-
cmd << " D#{
|
|
88
|
-
cmd << " J#{
|
|
86
|
+
cmd << " D#{delta}" if delta
|
|
87
|
+
cmd << " J#{initial}" if initial
|
|
89
88
|
# Always set a TTL if an initial value is specified
|
|
90
|
-
cmd << " N#{ttl
|
|
89
|
+
cmd << " N#{ttl || 0}" if ttl || initial
|
|
91
90
|
cmd << cas_string(cas)
|
|
92
91
|
cmd << ' q' if quiet
|
|
93
92
|
cmd << " M#{incr ? 'I' : 'D'}"
|
|
@@ -137,18 +136,6 @@ module Dalli
|
|
|
137
136
|
end
|
|
138
137
|
end
|
|
139
138
|
|
|
140
|
-
# Numeric flag values are written straight into the command line, so a
|
|
141
|
-
# value that isn't an integer -- e.g. a String carrying CRLF -- would let
|
|
142
|
-
# the caller inject further memcached commands (GHSA-6wmv-xq9m-fmp7).
|
|
143
|
-
# Converting to Integer means only digits ever reach the wire. Strings
|
|
144
|
-
# are parsed as base 10 so "010" means 10, as memcached would read it,
|
|
145
|
-
# rather than octal 8.
|
|
146
|
-
def self.integer_flag(name, value)
|
|
147
|
-
value.is_a?(String) ? Integer(value, 10) : Integer(value)
|
|
148
|
-
rescue ArgumentError, TypeError, FloatDomainError
|
|
149
|
-
raise ArgumentError, "#{name} must be an Integer, got #{value.inspect}"
|
|
150
|
-
end
|
|
151
|
-
|
|
152
139
|
def self.cas_string(cas)
|
|
153
140
|
cas = parse_to_64_bit_int(cas, nil)
|
|
154
141
|
cas.nil? || cas.zero? ? '' : " C#{cas}"
|
|
@@ -7,39 +7,38 @@ module Dalli
|
|
|
7
7
|
module Protocol
|
|
8
8
|
##
|
|
9
9
|
# Manages the buffer for responses from memcached.
|
|
10
|
-
# Uses an offset-based approach to avoid string allocations
|
|
11
|
-
# when advancing through parsed responses.
|
|
12
10
|
##
|
|
13
11
|
class ResponseBuffer
|
|
14
|
-
# Compact the buffer when the consumed portion exceeds this
|
|
15
|
-
# threshold and represents more than half the buffer
|
|
16
|
-
COMPACT_THRESHOLD = 4096
|
|
17
|
-
|
|
18
12
|
def initialize(io_source, response_processor)
|
|
19
13
|
@io_source = io_source
|
|
20
14
|
@response_processor = response_processor
|
|
21
15
|
@buffer = nil
|
|
22
|
-
@offset = 0
|
|
23
16
|
end
|
|
24
17
|
|
|
25
18
|
def read
|
|
26
19
|
@buffer << @io_source.read_nonblock
|
|
27
20
|
end
|
|
28
21
|
|
|
29
|
-
# Attempts to process a single response from the buffer
|
|
30
|
-
# advancing the
|
|
22
|
+
# Attempts to process a single response from the buffer. Starts
|
|
23
|
+
# by advancing the buffer to the specified start position
|
|
31
24
|
def process_single_getk_response
|
|
32
|
-
bytes, status, cas, key, value = @response_processor.getk_response_from_buffer(@buffer
|
|
33
|
-
|
|
34
|
-
compact_if_needed
|
|
25
|
+
bytes, status, cas, key, value = @response_processor.getk_response_from_buffer(@buffer)
|
|
26
|
+
advance(bytes)
|
|
35
27
|
[status, cas, key, value]
|
|
36
28
|
end
|
|
37
29
|
|
|
30
|
+
# Advances the internal response buffer by bytes_to_advance
|
|
31
|
+
# bytes. The
|
|
32
|
+
def advance(bytes_to_advance)
|
|
33
|
+
return unless bytes_to_advance.positive?
|
|
34
|
+
|
|
35
|
+
@buffer = @buffer.byteslice(bytes_to_advance..-1)
|
|
36
|
+
end
|
|
37
|
+
|
|
38
38
|
# Resets the internal buffer to an empty state,
|
|
39
39
|
# so that we're ready to read pipelined responses
|
|
40
40
|
def reset
|
|
41
41
|
@buffer = ''.b
|
|
42
|
-
@offset = 0
|
|
43
42
|
end
|
|
44
43
|
|
|
45
44
|
# Ensures the buffer is initialized for reading without discarding
|
|
@@ -49,30 +48,16 @@ module Dalli
|
|
|
49
48
|
return if in_progress?
|
|
50
49
|
|
|
51
50
|
@buffer = ''.b
|
|
52
|
-
@offset = 0
|
|
53
51
|
end
|
|
54
52
|
|
|
55
53
|
# Clear the internal response buffer
|
|
56
54
|
def clear
|
|
57
55
|
@buffer = nil
|
|
58
|
-
@offset = 0
|
|
59
56
|
end
|
|
60
57
|
|
|
61
58
|
def in_progress?
|
|
62
59
|
!@buffer.nil?
|
|
63
60
|
end
|
|
64
|
-
|
|
65
|
-
private
|
|
66
|
-
|
|
67
|
-
# Only compact when we've consumed a significant portion of the buffer.
|
|
68
|
-
# This avoids per-response string allocation while preventing unbounded
|
|
69
|
-
# memory growth for large pipelines.
|
|
70
|
-
def compact_if_needed
|
|
71
|
-
return unless @offset > COMPACT_THRESHOLD && @offset > @buffer.bytesize / 2
|
|
72
|
-
|
|
73
|
-
@buffer = @buffer.byteslice(@offset..)
|
|
74
|
-
@offset = 0
|
|
75
|
-
end
|
|
76
61
|
end
|
|
77
62
|
end
|
|
78
63
|
end
|