dalli 4.3.10 → 5.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (36) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +23 -161
  3. data/Gemfile +1 -1
  4. data/README.md +4 -55
  5. data/lib/dalli/client.rb +39 -98
  6. data/lib/dalli/compressor.rb +5 -53
  7. data/lib/dalli/instrumentation.rb +1 -1
  8. data/lib/dalli/key_manager.rb +2 -28
  9. data/lib/dalli/options.rb +2 -2
  10. data/lib/dalli/pipelined_getter.rb +32 -46
  11. data/lib/dalli/pipelined_setter.rb +8 -17
  12. data/lib/dalli/protocol/base.rb +31 -160
  13. data/lib/dalli/protocol/connection_manager.rb +20 -137
  14. data/lib/dalli/protocol/meta.rb +15 -54
  15. data/lib/dalli/protocol/{meta/request_formatter.rb → request_formatter.rb} +8 -21
  16. data/lib/dalli/protocol/response_buffer.rb +12 -27
  17. data/lib/dalli/protocol/{meta/response_processor.rb → response_processor.rb} +32 -64
  18. data/lib/dalli/protocol/server_config_parser.rb +1 -1
  19. data/lib/dalli/protocol/value_compressor.rb +7 -36
  20. data/lib/dalli/protocol/value_serializer.rb +1 -1
  21. data/lib/dalli/protocol.rb +9 -7
  22. data/lib/dalli/ring.rb +2 -2
  23. data/lib/dalli/servers_arg_normalizer.rb +1 -1
  24. data/lib/dalli/socket.rb +0 -29
  25. data/lib/dalli/version.rb +2 -2
  26. data/lib/dalli.rb +0 -3
  27. data/lib/rack/session/dalli.rb +2 -1
  28. metadata +7 -14
  29. data/lib/dalli/json_serializer.rb +0 -28
  30. data/lib/dalli/protocol/binary/request_formatter.rb +0 -116
  31. data/lib/dalli/protocol/binary/response_header.rb +0 -36
  32. data/lib/dalli/protocol/binary/response_processor.rb +0 -245
  33. data/lib/dalli/protocol/binary/sasl_authentication.rb +0 -60
  34. data/lib/dalli/protocol/binary.rb +0 -202
  35. data/lib/dalli/protocol_deprecations.rb +0 -45
  36. /data/lib/dalli/protocol/{meta/key_regularizer.rb → key_regularizer.rb} +0 -0
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 41490f01a85678506bed8f5e234cbbea7dadee8d5369f8c9800206a63a82260c
4
- data.tar.gz: 5176f61b9332988912f0216f516caedcfe0fb8a86f0eeea4d129e52ec7a6f183
3
+ metadata.gz: 16d7d5950137f4a51ae41392eaa576732595ac4020ae7be972d6fb4b33b9861d
4
+ data.tar.gz: ebb799522259a9a32a86e1dbfd13e810427f787f8110415509288655a5a5460b
5
5
  SHA512:
6
- metadata.gz: abccb3715898dcb8c1f6f3581f807253c91cf0190a2ff7ec44c70aea73d4312e5f39faebce3f510b47686cce02c8545806397cb8fb574801b36a6077a06582a2
7
- data.tar.gz: 468fa248795f1e7c4905bf47c365e0807428ca2b05e5499d4686ea94b506584d818c5bd03b09d822182def1b3cb76ac97feab8977b8a906d5362d891f7120185
6
+ metadata.gz: 74e32eace9001bd50be0617f10234f1440f86e506facc7247cf6041cf81fa85e966a5721e2a7ff364eb89da19bd46cc92b2631d99d4cbb21aee8240a475a76b5
7
+ data.tar.gz: 8e2bc294a1fa08a04f0bdfa300394751e63a7c1a21cac6aa1080a37e9491fe33091150047f664607ae42596d0bb2ac60f05c06ad97aee83165de754b9cceeed2
data/CHANGELOG.md CHANGED
@@ -1,178 +1,34 @@
1
1
  Dalli Changelog
2
2
  =====================
3
3
 
4
- Unreleased
4
+ 5.0.0
5
5
  ==========
6
6
 
7
- 4.3.10
8
- ==========
9
-
10
- Bug fixes:
11
-
12
- - `set_multi` across more than one server stored `nil` for a Symbol key or a key over 250 bytes, in place of the value given
13
-
14
- 4.3.9
15
- ==========
16
-
17
- Security:
18
-
19
- - Keep a connection shared by several threads or fibers in step during a `quiet` block (GHSA-5xvx-r8rg-5qm2)
20
- - A `quiet` (or `multi`) block is quiet only for the thread or fiber that runs it. memcached still replies to a quiet request that fails (for example an `add` of a key that exists), and another thread or fiber using the same client while the block was open read that reply as its own, so its later requests could return another key's value. A non-quiet request now reads any pending quiet replies first
21
- - Don't let a `get_multi` that stopped early leave replies for a later request (GHSA-5xvx-r8rg-5qm2)
22
- - A pipelined `get_multi` stopped by a non-network exception (its block raising or breaking, `Timeout.timeout`, or a value that can't be deserialized) left its unread replies in the response buffer, and a later `get_multi` large enough to use the interleaved path returned them as its own. The buffer is now cleared when a connection closes or a pipelined get starts, and unfinished pipelines are aborted
23
- - Affects every supported line (the `get_multi` issue, 4.3 and later), with both protocols; with the binary protocol the connection stayed out of step after the block ended. Fixed in 5.2.4, 5.1.6, 5.0.12, 4.3.9 and 3.2.15
24
-
25
- Bug fixes:
26
-
27
- - A per-request `raw: true` write no longer compresses the value (backport of #1213)
28
- - Values at or above the compression threshold (4 KB by default) were compressed and flagged even with `raw: true`. Raw reads ignore flags, so reading them back with `raw: true` returned the compressed bytes
29
- - Raw values are now stored exactly as given. Callers that pass `raw: true` without `compress: false` store large values uncompressed; Rails' `MemCacheStore` passes `compress: false` and isn't affected
30
- - Handle failed TLS handshakes like other connection failures, and warn when TLS verification is off (backport of #1215)
31
- - A failed or timed-out handshake left the TCP socket open until GC and raised `OpenSSL::SSL::SSLError` directly, so the server was never marked down. The socket is now closed, and the error goes through the usual retry and `socket_max_failures` handling, raising `Dalli::NetworkError` (or `Dalli::RingError` once no server is available)
32
- - Certificate and hostname checks come only from the `ssl_context` you pass, and a bare `OpenSSL::SSL::SSLContext.new` checks neither. Dalli now logs a warning once per process when `verify_mode` doesn't include `VERIFY_PEER` or `verify_hostname` is false; see the new TLS section in the README
33
- - A reply line longer than 8 KiB without a CRLF, or one cut short by the connection closing, is now a network error that closes the connection. Before, the line had no length limit (backport of #1216)
34
- - One value that can't be stored no longer discards other buffered writes (backport of #1218)
35
- - A value that raised `Dalli::MarshalError` or `Dalli::ValueOverMaxSize` closed the connection, which dropped writes still buffered from the same `quiet` block or `set_multi`, and `set_multi` did this without raising
36
- - `set_multi` now stores every other key, skips the bad one, and then raises
37
-
38
- 4.3.8
39
- ==========
40
-
41
- Security:
42
-
43
- - Don't rely on memcached's auto-create for keys that need base64 encoding (GHSA-r47x-6p6r-mvcp)
44
- - memcached stores an item made by auto-create (`ma` with `N`/`J`, or `mg` with `N`) for a base64-encoded key without its binary-key flag, and later replies name the item by its raw key bytes. A key containing spaces or CR/LF could then make a multi-get read the replies after it as different ones
45
- - `incr` and `decr` with an initial value now create such counters with an add instead. Return values are unchanged
46
- - The new `verify_reply_keys` option makes `get_multi` and `get_multi_cas` ignore replies for keys they didn't request, which guards against items created by older Dalli versions or other clients until memcached is upgraded. It's off by default, since it costs a multi-get about 15%
47
- - Affects `protocol: :meta` since 3.2.0; fixed in 5.2.3, 5.1.5, 5.0.11, 4.3.8 and 3.2.14. memcached 1.6.45 and earlier still name these items by their raw key bytes
48
-
49
- 4.3.7
50
- ==========
51
-
52
- Security:
53
-
54
- - Keep the caller's key when retrying a request (GHSA-m252-9cgf-vx2w)
55
- - With a `namespace`, a request retried after a transient network error (a timeout, or a connection closed by memcached or a proxy) applied the namespace a second time, so a retried read could return a different key's value and a retried write could overwrite a different key
56
- - Affects `get_with_metadata` and `fetch_with_lock` since 4.1.0 (single-key operations on 4.x aren't affected); fixed in 5.2.2, 5.1.4, 5.0.10 and 4.3.7. Clients without a namespace aren't affected
57
- - Complete the fix for per-request `raw: true` on reads (GHSA-wr87-m4jw-29x5)
58
- - `cas` and `cas!` (both protocols) and `fetch_with_lock` (meta protocol) accepted `raw: true` but didn't pass it to the read, so the value was still deserialized. They now honor it
59
- - A read made with `raw: true` doesn't ask for flags, but a reply carrying them anyway (from a proxy or a hostile server) still had its value deserialized. Meta protocol raw reads now ignore flags in the reply
60
- - Affects 4.3.6 and earlier; fixed in 5.2.2, 5.1.4, 5.0.10, 4.3.7 and 3.2.13
61
- - Complete the fork-safety fix (GHSA-w39f-xq2m-4g8x)
62
- - Fork detection used a pid cache refreshed by a `Process._fork` hook. When another library's fork hook ran first in the child (connection_pool, loaded before Dalli as in Rails, closes its connections there), Dalli didn't know it was in a forked child and ended the parent's TLS session. It now checks `Process.pid`
63
- - 4.3.6's write buffer kept a reference to the caller's value string with the meta protocol, so changing the string before the buffer was sent (for example, reusing one buffer inside a `quiet` block) changed the bytes sent, losing values or adding commands. The buffer now holds a copy
64
- - Fixed in 4.3.7 and 3.2.13
65
-
66
- Bug fixes:
67
-
68
- - `get_with_metadata` and `fetch_with_lock` retried the final error raised when a server is marked down, so with `down_retry_delay: 0` they retried an unresponsive server forever. They now retry only retryable errors, like other operations
69
- - Fix a regression in 4.3.6: a `get_multi` that didn't finish within `socket_timeout` counted toward `socket_max_failures`, so two slow `get_multi` calls in a row marked a healthy server down. It now just closes the connection
70
- - Fix a regression in 4.3.6: over TLS, every request buffered in a `quiet` block was sent as its own TLS record and system call (2.5 times slower for a block of 2000 deletes). Each flush is one write again
71
- - Handle malformed replies from a broken or hostile server or proxy: a negative value size, a hit with no key, or a hit with no `s` flag could leave the connection out of step or raise a non-Dalli error
72
- - With `decompressed_max_bytes` in effect, data after the end of a compressed value's stream was returned as part of the value. It's now ignored, as without the limit
73
- - With a `digest_class` whose digests aren't short hex strings, shortening a long key could loop forever. It now raises `ArgumentError`
74
-
75
- Development:
76
-
77
- - Run the Tests, RuboCop and Profiles workflows on pushes to `main` and the `*-stable` branches only, so a pull request's branch isn't tested twice (backport of #1198)
78
-
79
- 4.3.6
80
- ==========
81
-
82
- Security:
7
+ **Breaking Changes:**
83
8
 
84
- - Fix pipelined `get_multi` returning one key's value for another after an error reply (GHSA-p6pm-ch9v-44vx)
85
- - The pipelined reply parser took any reply without a body for the end of the batch. An error reply for one key (`CLIENT_ERROR`, `SERVER_ERROR`, or `EN` from a proxy) ended it early, and the replies still on the connection were read as the replies to later commands, so a `get` could return another key's value
86
- - Keys are now measured in bytes as sent (after base64 encoding when the meta protocol needs it). A key under 250 characters but over 250 bytes, for example one with many non-ASCII characters, passed validation:
87
- - with the meta protocol, memcached rejected it with the error reply above
88
- - with the binary protocol, memcached dropped the connection, and `get` or `get_multi` with that key retried forever
89
- - Such keys are now truncated like other long keys; keys that worked before are unchanged
90
- - The reply parser affects `protocol: :meta` since 3.2.0; the key length affects both protocols; fixed in 5.2.1, 5.1.3, 5.0.9, 4.3.6 and 3.2.12
91
- - Stop retrying an unresponsive server forever (GHSA-4qp6-2jcr-596v)
92
- - A server that accepted connections but never answered (or dropped the connection on a request) was retried forever, hanging the caller, because each successful reconnect reset the failure count. Requests now fail after `socket_max_failures` attempts and the server is marked down, as when it can't be reached at all; it gets a full set of attempts again after `down_retry_delay`
93
- - Affects all versions; fixed in 5.2.1, 5.1.3, 5.0.9, 4.3.6 and 3.2.12
94
- - Limit decompressed and reply value sizes (GHSA-3553-vcg5-72jw)
95
- - Values flagged as compressed were inflated without limit, so a small stored value could expand to gigabytes on read, whatever the client's `compress` or `serializer` settings. The new `decompressed_max_bytes` option (default 128 MiB; `nil` disables it) makes a read that would pass it raise `Dalli::UnmarshalError`. Custom compressors whose `decompress` takes only the data keep working, without the limit
96
- - The value size in a reply (a meta `VA` size or a binary body length) was used to read or buffer that many bytes, so a malicious server could make the client allocate gigabytes. Sizes over 1 GiB, memcached's largest item, now raise `Dalli::DalliError` before reading
97
- - Affects all versions; fixed in 5.2.1, 5.1.3, 5.0.9, 4.3.6 and 3.2.12
98
- - Honor per-request `raw: true` on reads, and add `Dalli::JSONSerializer` (GHSA-wr87-m4jw-29x5)
99
- - With the binary protocol, `get`, `gat` and `fetch`, and with the meta protocol, `get_with_metadata`, ignored a per-request `raw: true` and deserialized the value according to its stored flags, so a caller who asked for raw bytes could still have `Marshal.load` run on data someone else wrote to memcached. They now return the stored bytes
100
- - The `raw` part affects binary `get` and `gat` in all versions, and `get_with_metadata` since 4.2.0
101
- - `serializer: JSON` reads values with `JSON.load`, which on json gem versions before 3.0 creates an object of the class named in a stored `json_class` key when the json additions are loaded. The new `Dalli::JSONSerializer` reads with `JSON.parse` and only returns plain JSON types. The README and the Marshal security warning now recommend it; `serializer: JSON` itself is unchanged
102
- - The `serializer: JSON` part affects any version used with the json gem before 3.0
103
- - Keep a forked child from writing to or closing the parent's connection (GHSA-w39f-xq2m-4g8x)
104
- - A child forked while the parent had requests buffered but not yet sent, such as quiet writes inside a `quiet` block, sent them again when it closed the client or reconnected: quiet writes ran twice, and the parent's later replies could be read as the replies to other requests, returning one key's value for another. Dalli now buffers requests itself and a forked child discards them
105
- - Affects 4.2.0 and later; fixed in 5.2.1, 5.1.3, 5.0.9 and 4.3.6
106
- - With TLS, a forked child that closed the client (or reconnected after detecting the fork) sent a TLS close on the connection the parent was still using, ending the parent's session. A forked child now closes only its own copy of the socket
107
- - Affects all versions with TLS; fixed in 5.2.1, 5.1.3, 5.0.9, 4.3.6 and 3.2.12
9
+ - **Removed binary protocol** - The meta protocol is now the only supported protocol
10
+ - The `:protocol` option is no longer used
11
+ - Requires memcached 1.6+ (for meta protocol support)
12
+ - Users on older memcached versions must upgrade or stay on Dalli 4.x
108
13
 
109
- Bug fixes:
14
+ - **Removed SASL authentication** - The meta protocol does not support authentication
15
+ - Use network-level security (firewall rules, VPN) or memcached's TLS support instead
16
+ - Users requiring SASL authentication must stay on Dalli 4.x with binary protocol
110
17
 
111
- - An exception raised by a `get_multi` block (such as `Timeout::Error`) was treated as a network failure: swallowed, with the whole `get_multi` retried and keys yielded twice. It now reaches the caller
112
- - Fix `set_multi` with the binary protocol
113
- - Each quiet `setq` waited for a reply memcached never sends for a successful write, so `set_multi` timed out on its first key without storing anything (and, with retries, hung or marked the server down). Affects 4.2.0 and later
114
-
115
- Notes:
116
-
117
- - Document [Ruby bug 21195](https://bugs.ruby-lang.org/issues/21195), a VM crash (`[BUG] rb_sys_fail_path_in(io_fillbuf, ...) - errno == 0`) when a socket read using `IO#timeout` is interrupted by a signal, in the README (#1188)
118
- - Affects Ruby 3.2.x, 3.3.0–3.3.7 and 3.4.0–3.4.2. Fixed in 3.3.8 and 3.4.3; Ruby 3.2 reached end of life without the fix
119
- - Ruby 3.1 doesn't use `IO#timeout` and isn't affected
120
-
121
- Development:
122
-
123
- - Drop Ruby 3.2 from the CI matrix, since that bug intermittently crashed its test runs (#1188)
124
- - Fix three flaky tests: the `KeyManager` namespace key-length test, the Rack session freshness test, and `MemcachedManager`'s start and stop handling (backport of #1141, #1182)
125
- - Fix flaky failover tests: move their ports out of Linux's ephemeral port range, and wait for memcached to accept connections after starting it (backport of #1184)
126
-
127
- 4.3.5
128
- ==========
129
-
130
- Bug fixes:
131
-
132
- - Fix multi-server `get_multi` stopping at an empty value (#1170)
133
- - The pipelined reply parser took a hit on a zero-length value (`VA 0`) for the terminating `MN`, so that server's remaining keys were silently missing from the result
134
- - Its value terminator and any replies not yet parsed stayed on the connection and were read as the replies to later commands on it. Those commands could fail with `Dalli::DalliError: Response error`, or a single-key `get` could silently return another key's value
135
- - An empty value is now returned as `''`, like a single-key `get` and single-server `get_multi`
136
- - Affects only clients using `protocol: :meta`; the default binary protocol is not affected
137
- - Thanks to Julian Richard Contreras for this contribution
138
-
139
- Development:
140
-
141
- - Fix offenses reported by RuboCop 1.91 and require `rubocop >= 1.91` (backport of #1162)
142
-
143
- 4.3.4
144
- ==========
145
-
146
- Security:
147
-
148
- - Fix memcached command injection through numeric arguments with the meta protocol (GHSA-6wmv-xq9m-fmp7)
149
- - With `protocol: :meta`, the `default` argument of `incr`/`decr`, and `fetch_with_lock`'s `lock_ttl` and `recache_threshold`, were written into the command without conversion, so a String containing CRLF injected additional memcached commands (e.g. `set`, `flush_all`) on the connection. The default binary protocol is not affected
150
- - These arguments must now be Integers, or Strings of decimal digits; anything else raises `ArgumentError` before a request is sent
151
- - As defense in depth, the meta `RequestFormatter` now converts every numeric flag it writes (`D`, `J`, `N`, `R`, `T`) to an Integer
152
- - Thanks to oss-security-shop for the report
153
-
154
- 4.3.3
155
- ==========
18
+ - **Ruby 3.3+ required** - Dropped support for Ruby 3.1 and 3.2
19
+ - Ruby 3.2 reached end-of-life in March 2026
20
+ - JRuby remains supported
156
21
 
157
22
  Performance:
158
23
 
159
- - Reduce object allocations in pipelined get response processing (#1072)
160
- - Offset-based `ResponseBuffer`: track a read offset instead of slicing a new string after every parsed response; compact only when the consumed portion exceeds 4KB and more than half the buffer
161
- - Inline response processor parsing: avoid intermediate array allocations from `split`-based header parsing in both binary and meta protocols
162
- - Block-based `pipeline_next_responses`: yield `(key, value, cas)` directly when a block is given, avoiding per-call Hash allocation
163
- - `PipelinedGetter`: replace Hash-based socket-to-server mapping with linear scan (faster for typical 1-5 server counts); use `Process.clock_gettime(CLOCK_MONOTONIC)` instead of `Time.now`
164
- - Add cross-version benchmark script (`bin/compare_versions`) for reproducible performance comparisons across Dalli versions
165
-
166
- Bug Fixes:
167
-
168
- - Skip OTel integration tests when meta protocol is unavailable (#1072)
169
-
170
- 4.3.2
171
- ==========
24
+ - **~7% read performance improvement** (CRuby only)
25
+ - Use native `IO#read` instead of custom `readfull` implementation
26
+ - Enabled by Ruby 3.3's `IO#timeout=` support
27
+ - JRuby continues to use `readfull` for compatibility
172
28
 
173
29
  OpenTelemetry:
174
30
 
175
- - Migrate to stable OTel semantic conventions
31
+ - Migrate to stable OTel semantic conventions (#1070)
176
32
  - `db.system` renamed to `db.system.name`
177
33
  - `db.operation` renamed to `db.operation.name`
178
34
  - `server.address` now contains hostname only; `server.port` is a separate integer attribute
@@ -182,6 +38,12 @@ OpenTelemetry:
182
38
  - Add `peer.service` span attribute
183
39
  - `:otel_peer_service` option for logical service naming
184
40
 
41
+ Internal:
42
+
43
+ - Simplified protocol directory structure: moved `lib/dalli/protocol/meta/*` to `lib/dalli/protocol/`
44
+ - Removed deprecated binary protocol files and SASL authentication code
45
+ - Removed `require 'set'` (autoloaded in Ruby 3.3+)
46
+
185
47
  4.3.1
186
48
  ==========
187
49
 
data/Gemfile CHANGED
@@ -18,7 +18,7 @@ group :development, :test do
18
18
  gem 'rack', '~> 3'
19
19
  gem 'rack-session'
20
20
  gem 'rake', '~> 13.0'
21
- gem 'rubocop', '>= 1.91' # disable-next directives require 1.91+
21
+ gem 'rubocop'
22
22
  gem 'rubocop-minitest'
23
23
  gem 'rubocop-performance'
24
24
  gem 'rubocop-rake'
data/README.md CHANGED
@@ -10,38 +10,14 @@ Dalli supports:
10
10
  * Fine-grained control of data serialization and compression
11
11
  * Thread-safe operation (either through use of a connection pool, or by using the Dalli client in threadsafe mode)
12
12
  * SSL/TLS connections to memcached
13
- * SASL authentication
14
13
  * OpenTelemetry distributed tracing (automatic when SDK is present)
15
14
 
16
15
  The name is a variant of Salvador Dali for his famous painting [The Persistence of Memory](http://en.wikipedia.org/wiki/The_Persistence_of_Memory).
17
16
 
18
17
  ## Requirements
19
18
 
20
- * Ruby 3.1 or later
21
- * memcached 1.4 or later (1.6+ recommended for meta protocol support)
22
-
23
- ### Known Ruby issue: crash on an interrupted socket read
24
-
25
- On Ruby 3.2 and later, Dalli sets socket timeouts with `IO#timeout`. Ruby 3.2.x, 3.3.0–3.3.7 and 3.4.0–3.4.2 have a bug ([Ruby #21195](https://bugs.ruby-lang.org/issues/21195)) where a timed read interrupted by a signal aborts the whole process:
26
-
27
- ```
28
- [BUG] rb_sys_fail_path_in(io_fillbuf, fd:N ) - errno == 0
29
- ```
30
-
31
- The crash is inside Ruby itself, so it can't be rescued. It was fixed in Ruby 3.3.8 and 3.4.3. Ruby 3.2 reached end of life without the fix. If you see this crash, upgrade to Ruby 3.3.8+ or 3.4.3+. Ruby 3.1 doesn't use `IO#timeout` and isn't affected.
32
-
33
- Ruby 3.2 is no longer part of this branch's CI matrix for the same reason.
34
-
35
- ## Protocol Options
36
-
37
- Dalli supports two protocols for communicating with memcached:
38
-
39
- * `:binary` (default) - Works with all memcached versions, supports SASL authentication
40
- * `:meta` - Requires memcached 1.6+, better performance for some operations, no authentication support
41
-
42
- ```ruby
43
- Dalli::Client.new('localhost:11211', protocol: :meta)
44
- ```
19
+ * Ruby 3.3 or later (JRuby also supported)
20
+ * memcached 1.6 or later
45
21
 
46
22
  ## Configuration Options
47
23
 
@@ -73,37 +49,10 @@ The separator must be a single non-alphanumeric character. Valid examples: `:`,
73
49
  By default, Dalli uses Ruby's Marshal for serialization. Deserializing untrusted data with Marshal can lead to remote code execution. If you cache user-controlled data, consider using a safer serializer:
74
50
 
75
51
  ```ruby
76
- Dalli::Client.new('localhost:11211', serializer: Dalli::JSONSerializer)
77
- ```
78
-
79
- `Dalli::JSONSerializer` reads values with `JSON.parse`, so it only ever returns plain hashes, arrays, strings, numbers, booleans and `nil`. Passing the `JSON` module itself (`serializer: JSON`) reads values with `JSON.load`, which can create objects of other classes from a stored `json_class` key and is not recommended for data an attacker might write.
80
-
81
- Per-request `raw: true` returns the stored bytes without deserializing or decompressing them, on every read method that accepts it (`get`, `get_with_metadata` and `fetch`).
82
-
83
- See the [4.0-Upgrade.md](4.0-Upgrade.md) guide for more information.
84
-
85
- ### Counters and verify_reply_keys
86
-
87
- memcached names an item made by auto-create for a base64-encoded key (one with whitespace, control or non-ASCII characters) by its raw key bytes in later replies (with `protocol: :meta`). Since 4.3.8, Dalli creates counters for such keys with an add rather than auto-create. Items created before then, or by other clients, can still be named that way until memcached releases a fix (1.6.45 and earlier behave this way). If keys passed to `incr` or `decr` with an initial value can contain untrusted input, set `verify_reply_keys: true` until then: multi-gets then ignore replies for keys they didn't request, at a cost of about 15% per multi-get.
88
-
89
- ## TLS
90
-
91
- Pass an `OpenSSL::SSL::SSLContext` as `ssl_context:` to connect over TLS. Certificate and hostname verification come only from that context: Dalli adds no checks of its own. A bare `OpenSSL::SSL::SSLContext.new` verifies nothing, so it accepts any certificate from any server. Configure the context to verify both:
92
-
93
- ```ruby
94
- ssl_context = OpenSSL::SSL::SSLContext.new
95
- ssl_context.set_params(
96
- verify_mode: OpenSSL::SSL::VERIFY_PEER,
97
- verify_hostname: true,
98
- ca_file: '/path/to/ca.crt' # omit to use the system's trusted certificates
99
- )
100
-
101
- Dalli::Client.new('memcached.example.com:11211', ssl_context: ssl_context)
52
+ Dalli::Client.new('localhost:11211', serializer: JSON)
102
53
  ```
103
54
 
104
- Dalli logs a warning once per process when the context's `verify_mode` is `VERIFY_NONE`, or when it is `VERIFY_PEER` but `verify_hostname` is false.
105
-
106
- A failed TLS handshake counts as a failed connection attempt, like a refused connection: once `socket_max_failures` attempts have failed, the server is marked down and requests fail over to the other servers.
55
+ See the [5.0-Upgrade.md](5.0-Upgrade.md) guide for upgrade information.
107
56
 
108
57
  ## OpenTelemetry Tracing
109
58
 
data/lib/dalli/client.rb CHANGED
@@ -1,8 +1,8 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  require 'digest/md5'
4
- require 'set'
5
4
 
5
+ # encoding: ascii
6
6
  module Dalli
7
7
  ##
8
8
  # Dalli::Client is the main class which developers will use to interact with
@@ -42,30 +42,22 @@ module Dalli
42
42
  # #fetch operations.
43
43
  # - :raw - If set, disables serialization and compression entirely at the client level.
44
44
  # Only String values are supported. This is useful when the caller handles its own
45
- # serialization (e.g., Rails' ActiveSupport::Cache). The per-request :raw option
46
- # (e.g. set(key, value, ttl, raw: true)) does the same for a single call: String
47
- # values are stored as-is, uncompressed, and non-String values raise
48
- # Dalli::MarshalError.
45
+ # serialization (e.g., Rails' ActiveSupport::Cache). Note: this is different from
46
+ # the per-request :raw option which converts values to strings but still uses the
47
+ # serialization pipeline.
49
48
  # - :digest_class - defaults to Digest::MD5, allows you to pass in an object that responds to the hexdigest method,
50
49
  # useful for injecting a FIPS compliant hash object.
51
- # - :protocol - one of either :binary or :meta, defaulting to :binary. This sets the protocol that Dalli uses
52
- # to communicate with memcached.
53
50
  # - :otel_db_statement - controls the +db.query.text+ span attribute when OpenTelemetry is loaded.
54
51
  # +:include+ logs the full operation and key(s), +:obfuscate+ replaces keys with "?",
55
52
  # +nil+ (default) omits the attribute entirely.
56
53
  # - :otel_peer_service - when set, adds a +peer.service+ span attribute with this value for logical service naming.
57
- # - :verify_reply_keys - defaults to false. When true, get_multi and get_multi_cas only accept replies for the keys
58
- # they requested; after a reply naming another key, the rest of that server's replies are
59
- # treated as misses. With the meta protocol, memcached can name an item by its raw key bytes
60
- # when the item was created by auto-create from a base64-encoded key, which Dalli no longer
61
- # does. Costs a multi-get about 15%.
62
54
  #
63
55
  def initialize(servers = nil, options = {})
64
56
  @normalized_servers = ::Dalli::ServersArgNormalizer.normalize_servers(servers)
65
57
  @options = normalize_options(options)
58
+ warn_removed_options(@options)
66
59
  @key_manager = ::Dalli::KeyManager.new(@options)
67
60
  @ring = nil
68
- emit_deprecation_warnings
69
61
  end
70
62
 
71
63
  #
@@ -107,10 +99,7 @@ module Dalli
107
99
  end
108
100
 
109
101
  ##
110
- # Get value with extended metadata using the meta protocol.
111
- #
112
- # IMPORTANT: This method requires memcached 1.6+ and the meta protocol (protocol: :meta).
113
- # It will raise an error if used with the binary protocol.
102
+ # Get value with extended metadata.
114
103
  #
115
104
  # @param key [String] the cache key
116
105
  # @param options [Hash] options controlling what metadata to return
@@ -138,20 +127,14 @@ module Dalli
138
127
  # # => { value: "data", cas: 123, hit_before: true, last_access: 42 }
139
128
  #
140
129
  def get_with_metadata(key, options = {})
141
- raise_unless_meta_protocol!
142
-
143
- # A retry re-runs this method with its original arguments, so `key` must
144
- # stay as the caller gave it; reassigning it would add the namespace again.
145
- validated_key = @key_manager.validate_key(key.to_s)
130
+ key = key.to_s
131
+ key = @key_manager.validate_key(key)
146
132
 
147
- server = ring.server_for_key(validated_key)
148
- Instrumentation.trace('get_with_metadata', trace_attrs('get_with_metadata', validated_key, server)) do
149
- server.request(:meta_get, validated_key, options)
133
+ server = ring.server_for_key(key)
134
+ Instrumentation.trace('get_with_metadata', trace_attrs('get_with_metadata', key, server)) do
135
+ server.request(:meta_get, key, options)
150
136
  end
151
- # Only a retryable error is retried. The NetworkError raised when a server
152
- # is marked down is final, as in #perform; retrying it looped forever once
153
- # down_retry_delay let the same server be tried again.
154
- rescue RetryableNetworkError => e
137
+ rescue NetworkError => e
155
138
  Dalli.logger.debug { e.inspect }
156
139
  Dalli.logger.debug { 'retrying get_with_metadata with new server' }
157
140
  retry
@@ -161,7 +144,7 @@ module Dalli
161
144
  # Fetch multiple keys efficiently.
162
145
  # If a block is given, yields key/value pairs one at a time.
163
146
  # Otherwise returns a hash of { 'key' => 'value', 'key2' => 'value1' }
164
- # rubocop:disable-next Style/ExplicitBlockArgument
147
+ # rubocop:disable Style/ExplicitBlockArgument
165
148
  def get_multi(*keys)
166
149
  keys.flatten!
167
150
  keys.compact!
@@ -173,6 +156,7 @@ module Dalli
173
156
  get_multi_hash(keys)
174
157
  end
175
158
  end
159
+ # rubocop:enable Style/ExplicitBlockArgument
176
160
 
177
161
  ##
178
162
  # Fetch multiple keys efficiently, including available metadata such as CAS.
@@ -216,9 +200,6 @@ module Dalli
216
200
  # cache entry (the "thundering herd" problem). Only one client wins the right to
217
201
  # regenerate; other clients receive the stale value (if available) or wait.
218
202
  #
219
- # IMPORTANT: This method requires memcached 1.6+ and the meta protocol (protocol: :meta).
220
- # It will raise an error if used with the binary protocol.
221
- #
222
203
  # @param key [String] the cache key
223
204
  # @param ttl [Integer] time-to-live for the cached value in seconds
224
205
  # @param lock_ttl [Integer] how long the lock/stub lives (default: 30 seconds)
@@ -244,21 +225,14 @@ module Dalli
244
225
  def fetch_with_lock(key, ttl: nil, lock_ttl: 30, recache_threshold: nil, req_options: nil, &block)
245
226
  raise ArgumentError, 'Block is required for fetch_with_lock' unless block_given?
246
227
 
247
- raise_unless_meta_protocol!
248
- validate_integer!(:lock_ttl, lock_ttl)
249
- validate_integer!(:recache_threshold, recache_threshold)
250
-
251
- # Left unchanged for the retry below; see get_with_metadata
252
- validated_key = @key_manager.validate_key(key.to_s)
228
+ key = key.to_s
229
+ key = @key_manager.validate_key(key)
253
230
 
254
- server = ring.server_for_key(validated_key)
255
- Instrumentation.trace('fetch_with_lock', trace_attrs('fetch_with_lock', validated_key, server)) do
256
- fetch_with_lock_request(validated_key, ttl, lock_ttl, recache_threshold, req_options, &block)
231
+ server = ring.server_for_key(key)
232
+ Instrumentation.trace('fetch_with_lock', trace_attrs('fetch_with_lock', key, server)) do
233
+ fetch_with_lock_request(key, ttl, lock_ttl, recache_threshold, req_options, &block)
257
234
  end
258
- # Only a retryable error is retried. The NetworkError raised when a server
259
- # is marked down is final, as in #perform; retrying it looped forever once
260
- # down_retry_delay let the same server be tried again.
261
- rescue RetryableNetworkError => e
235
+ rescue NetworkError => e
262
236
  Dalli.logger.debug { e.inspect }
263
237
  Dalli.logger.debug { 'retrying fetch_with_lock with new server' }
264
238
  retry
@@ -307,11 +281,6 @@ module Dalli
307
281
  # Any error replies will be discarded at the end of the block, and
308
282
  # Dalli client methods invoked inside the block will not
309
283
  # have return values
310
- #
311
- # The block is quiet only for the thread or fiber that runs it. Another
312
- # thread or fiber sharing this client can make requests meanwhile; a
313
- # non-quiet one first sends the block's buffered requests and discards
314
- # their replies, so it reads only its own.
315
284
  def quiet
316
285
  old = Thread.current[::Dalli::QUIET]
317
286
  Thread.current[::Dalli::QUIET] = true
@@ -420,8 +389,6 @@ module Dalli
420
389
  ##
421
390
  # Incr adds the given amount to the counter on the memcached server.
422
391
  # Amt must be a positive integer value.
423
- # Default, if given, must be an Integer (or a String of decimal digits);
424
- # anything else raises ArgumentError.
425
392
  #
426
393
  # If default is nil, the counter must already exist or the operation
427
394
  # will fail and will return nil. Otherwise this method will return
@@ -434,7 +401,6 @@ module Dalli
434
401
  # If the value already exists, it must have been set with raw: true
435
402
  def incr(key, amt = 1, ttl = nil, default = nil)
436
403
  check_positive!(amt)
437
- validate_integer!(:default, default)
438
404
 
439
405
  perform(:incr, key, amt.to_i, ttl_or_default(ttl), default)
440
406
  end
@@ -442,8 +408,6 @@ module Dalli
442
408
  ##
443
409
  # Decr subtracts the given amount from the counter on the memcached server.
444
410
  # Amt must be a positive integer value.
445
- # Default, if given, must be an Integer (or a String of decimal digits);
446
- # anything else raises ArgumentError.
447
411
  #
448
412
  # memcached counters are unsigned and cannot hold negative values. Calling
449
413
  # decr on a counter which is 0 will just return 0.
@@ -459,7 +423,6 @@ module Dalli
459
423
  # If the value already exists, it must have been set with raw: true
460
424
  def decr(key, amt = 1, ttl = nil, default = nil)
461
425
  check_positive!(amt)
462
- validate_integer!(:default, default)
463
426
 
464
427
  perform(:decr, key, amt.to_i, ttl_or_default(ttl), default)
465
428
  end
@@ -538,10 +501,6 @@ module Dalli
538
501
 
539
502
  private
540
503
 
541
- # Records hit/miss metrics on a span for cache observability.
542
- # @param span [OpenTelemetry::Trace::Span, nil] the span to record on
543
- # @param key_count [Integer] total keys requested
544
- # @param hit_count [Integer] keys found in cache
545
504
  def record_hit_miss_metrics(span, key_count, hit_count)
546
505
  return unless span
547
506
 
@@ -601,17 +560,8 @@ module Dalli
601
560
  raise ArgumentError, "Positive values only: #{amt}" if amt.negative?
602
561
  end
603
562
 
604
- # Numeric arguments that become meta protocol flags (GHSA-6wmv-xq9m-fmp7).
605
- # Checked before the request starts so a bad value raises a clean
606
- # ArgumentError, instead of Protocol::Base#request logging it as
607
- # unexpected and closing the connection.
608
- def validate_integer!(name, value)
609
- Dalli::Protocol::Meta::RequestFormatter.integer_flag(name, value) unless value.nil?
610
- end
611
-
612
563
  def cas_core(key, always_set, ttl = nil, req_options = nil)
613
- # req_options reaches the read too, so raw: true returns the stored bytes
614
- (value, cas) = perform(:cas, key, req_options)
564
+ (value, cas) = perform(:cas, key)
615
565
  return if value.nil? && !always_set
616
566
 
617
567
  newvalue = yield(value)
@@ -620,12 +570,7 @@ module Dalli
620
570
 
621
571
  def fetch_with_lock_request(key, ttl, lock_ttl, recache_threshold, req_options)
622
572
  server = ring.server_for_key(key)
623
- # req_options (e.g. raw: true) reaches the read too. It's the base, not
624
- # the override: fetch_with_lock's own lock_ttl/recache_threshold win.
625
- meta_options = req_options.is_a?(Hash) ? req_options.dup : {}
626
- meta_options[:vivify_ttl] = lock_ttl
627
- meta_options[:recache_ttl] = recache_threshold
628
- result = server.request(:meta_get, key, meta_options)
573
+ result = server.request(:meta_get, key, { vivify_ttl: lock_ttl, recache_ttl: recache_threshold })
629
574
 
630
575
  return result[:value] unless result[:won_recache]
631
576
 
@@ -645,16 +590,7 @@ module Dalli
645
590
  end
646
591
 
647
592
  def ring
648
- @ring ||= Dalli::Ring.new(@normalized_servers, protocol_implementation, @options)
649
- end
650
-
651
- def protocol_implementation
652
- @protocol_implementation ||= case @options[:protocol]&.to_s
653
- when 'meta'
654
- Dalli::Protocol::Meta
655
- else
656
- Dalli::Protocol::Binary
657
- end
593
+ @ring ||= Dalli::Ring.new(@normalized_servers, @options)
658
594
  end
659
595
 
660
596
  ##
@@ -665,7 +601,7 @@ module Dalli
665
601
  #
666
602
  # This method also forces retries on network errors - when
667
603
  # a particular memcached instance becomes unreachable, or the
668
- # operational times out.
604
+ # operation times out.
669
605
  ##
670
606
  def perform(*all_args)
671
607
  return yield if block_given?
@@ -692,6 +628,21 @@ module Dalli
692
628
  raise ArgumentError, "cannot convert :expires_in => #{opts[:expires_in].inspect} to an integer"
693
629
  end
694
630
 
631
+ REMOVED_OPTIONS = {
632
+ protocol: 'Dalli 5.0 only supports the meta protocol. The :protocol option has been removed.',
633
+ username: 'Dalli 5.0 removed SASL authentication support. The :username option is ignored.',
634
+ password: 'Dalli 5.0 removed SASL authentication support. The :password option is ignored.'
635
+ }.freeze
636
+ private_constant :REMOVED_OPTIONS
637
+
638
+ def warn_removed_options(opts)
639
+ REMOVED_OPTIONS.each do |key, message|
640
+ next unless opts.key?(key)
641
+
642
+ Dalli.logger.warn(message)
643
+ end
644
+ end
645
+
695
646
  def pipelined_getter
696
647
  PipelinedGetter.new(ring, @key_manager)
697
648
  end
@@ -703,15 +654,5 @@ module Dalli
703
654
  def pipelined_deleter
704
655
  PipelinedDeleter.new(ring, @key_manager)
705
656
  end
706
-
707
- def raise_unless_meta_protocol!
708
- return if protocol_implementation == Dalli::Protocol::Meta
709
-
710
- raise Dalli::DalliError,
711
- 'This operation requires the meta protocol (memcached 1.6+). ' \
712
- 'Use protocol: :meta when creating the client.'
713
- end
714
-
715
- include ProtocolDeprecations
716
657
  end
717
658
  end